dwrz/talks
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
2334f4eaf0281968d2b4f16da7f821b9c4f2758b
talks/concurrency/slides.org
T
dwrz 2334f4eaf0 Add concurrency
2026-10-27 12:41:06 +00:00

12 KiB
Raw Blame History

1. Concurrency

Slides and code: https://github.com/dwrz/talks/concurrency.

2. Why

You'll likely encounter it at work:

  • Client requests to your backend
  • Database and API requests made by your backend
  • Frontend rendering

3. Why

Performance Benefits

  • I/O versus CPU bound applications.
  • Typically outweigh gains from optimized data structures and algorithms.

  • Hardware limitations.

    • Rate of single core performance improvement is slowing.
    • More cores.

4. Why

Expressive

5. Why

Difficult to do correctly.

Easy to do incorrectly.

6. Goals

  • Give you a rough sense of what concurrency is.
  • Provide context to help you understand how it's made possible.
  • Practical demonstration, to expose some of the trade-offs.

7. Three Parts

  1. Fundamentals
  2. Application
  3. Infrastructure

8. Definition

Concurrency

  1. A running together in place or time.
  2. Simultaneous occurrence.

OK starting point, but:

Time, and simultaneity, is complicated.

A second is a duration, not an instant.

Two things can happen in the same second and not be simultaneous.

In computer science, this nuance matters.

9. Time

System Event Actual Latency Scaled Latency
One CPU cycle 0.4 ns 1 s
Level 1 cache access 0.9 ns 2 s
Level 2 cache access 2.8 ns 7 s
Level 3 cache access 28 ns 1 min
Main memory access ~100 ns 4 min
NVMe SSD I/O ~25 μs 17 hrs
SSD I/O 50–150 μs 1.5–4 days
Rotational disk I/O 1–10 ms 1–9 months
Internet: SF to NYC 65 ms 5 years
Internet: SF to HK 141 ms 11 years

Source: https://www.prowesscorp.com/computer-latency-at-a-human-scale/

10. Example

  • Get the current temperature of ten most populous cities in the US.
  • First, sequentially.
  • Then, concurrently.

cmd/temperatures/run.sh

11. Example

images/concurrent-requests.jpg

12. Cooking Recipes

  • Step-by-step

  • Concurrent

    • Pre-heat oven
    • Boil water
    • Timers

13. Multitasking

Computers can "simultaneously" work on:

  • Different parts of a recipe
  • Different recipes

14. htop

15. How?

  • Operating System's Scheduler
  • Hardware (interrupts)
  • Not just multiple cores!

16. NB: Concurrency != Parallelism

First consumer multi-core CPUs released in mid-2000's.

Even before that, computers were able to multitask with one core.

17. Cooperative Scheduling

images/cooperative-scheduling.jpg

18. Preemptive Scheduling

images/preemptive-scheduling.jpg

19. Preemptive Scheduling

images/preemptive-scheduling-tree.jpg

20. Single Core

images/single-core-concurrency.jpg

21. Multi Core

images/preemptive-scheduling-multi-core.jpg

22. Concurrency versus Parallelism

Parallelism

performing computations at the same moment.

  • Multiple cores.
  • Multiples computers.
  • Kind of like having multiple cooks in a kitchen.
Concurrency

performing computations over inter-leaving, non-sequential time periods.

  • Single or multi-core.

23. Processes and Threads

Computers can "simultaneously" work on:

  • Different parts of a recipe
  • Different recipes
  • Process :: a program in execution (recipe).
  • Thread :: smallest sequence of programmed instructions that can be managed independently by a scheduler (part of a recipe).

24. Processes and Threads

Processes

  • Have their own memory space

    • Instructions
    • Constants
    • Stack
    • Heap
  • Start out with one thread (the main thread).
  • Can fork – create a child process.
  • Inter-Process Communication (IPC)

25. Processes and Threads

Threads (a.k.a, lightweight processes)

  • Share:

    • Memory
    • Instructions
    • Open files (descriptors)
    • Signals and signal handlers
    • Etcetera…
  • Less resource intensive than processes.

26. Processes and Threads

images/process-vs-thread.jpg

Source: https://stackoverflow.com/questions/16354460/forking-vs-threading/16354658

27. htop

Press H to show threads.

28. Languages and Runtimes

  • C, C++, Rust allow for fine-grained control of processes and threads.
  • JavaScript: single-threaded (for your code), runtimes (Chrome, nodejs) implement concurrency via an event loop.
  • Go: runtime multi-threading, multiplexes goroutines onto threads.

29. nodejs

images/nodejs-event-loop.png

Source: https://medium.com/preezma/node-js-event-loop-architecture-go-deeper-node-core-c96b4cec7aa4

30. Go

images/go-scheduler.png

Source: https://freethreads.net/2019/01/23/go-runtime-scheduler-design-internals/

31. Quick Review

  • Sequential versus Concurrent Steps

    • Compare with recipes.

  • Concurrency versus Parallelism

    • Multiple cooks.
  • Operating System Scheduler
  • Processes and Threads
  • Languages and Runtimes

32. Part 2: Using Concurrency

Typical use cases:

  • Network I/O

    • Requests between clients and servers.
    • Requests to a database.
    • Requests between microservices.
    • Requests to external APIs.

  • User interfaces

    • Capturing user input
    • Rendering output
  • Similar, but independent computations.
  • Solving problems that are best expressed with concurrency.

33. JavaScript

  • Callbacks
  • Promises
  • async/await

What is the output of the following? Why?

console.log('start');

setTimeout(() => console.log('callback 1'), 0);

console.log('middle');

setTimeout(() => console.log('callback 2'), 0);

console.log('end');

34. Go

  1. Shared Memory with Mutex (Mutual Exclusion)

    • Pitfall: synchronization, debugging.

  2. Message Passing (Channels and Goroutines)

    • Pitfall: resources usage, complexity.

35. Mutex Example

Concurrent updates to a bank account balance.

Code: cmd/mutex/main.go

Run: cmd/mutex/run.sh

  • Without mutex.
  • With mutex.

36. Data Races

images/data-race.jpg

37. Mutex

Mutual Exclusion

  • Purpose: protect a critical section of code.
  • Effect: serialization.

38. Race Detector

  • Without mutex.
  • With mutex.

39. Tradeoffs: Validating Correctness, Testing, Debugging

  • Non-deterministic behavior

    • Works fine on one run, fatal bug in another.
    • Up to how the scheduler(s) order things at execution time.

  • Concealed bugs

    • Heisenbugs

40. Message Passing

Proverbs:

Don't communicate by sharing memory, share memory by communicating.

Channels orchestrate, mutexes serialize.

41. Requirements

  • Read CSV

    • 50 in our example
    • Full list is ~42,000

  • Get temperature with OpenWeatherMap API

    • 60 requests per minute
    • Need to rate limit our API requests.
  • Display running list of top ten temperatures.

42. Concurrency Hallmarks

  • Repeatedly processing similar data (lines of CSV)
  • Network I/O
  • UI I/O

43. Message Passing Example

Run: cmd/csp/run.sh

44. Code Walkthrough

cmd/csp/main.go

45. Channels and Goroutines

images/message-passing-example-diagram.jpg

46. Channels and Goroutines

images/go-scheduler.jpg

47. Scaling

What if:

  • We wanted to process the entire list of 42,000 zip codes?
  • We had access to the Enterprise tier of the Open Weather Map API (200,000 requests per minute).

Could we just get rid of the limiter?

48. Resources

Probably not, because:

  • OS limit on open files (ulimit)

    • Ask me how I know…
  • Server load
  • Network load

  • CPU

    • Ranking goroutine becomes a chokepoint.
    • Data structures and algorithms could help here.
  • Goroutines are cheap, but not free.

49. Possible Solutions

  • Use a semaphore to limit concurrency.
  • Buffers

50. Designing and Managing Concurrency

  • Size of input matters.
  • Resources
  • Pipelines
  • Chokepoints
  • Backpressure
  • Complexity

51. Quick Review

Techniques for implementing and managing concurrency:

  • Event loop
  • Mutexes
  • Communication (channels and goroutines)

Concurrency Pitfalls:

  • Synchronization
  • Resource usage
  • Debugging

  • Complexity

    • Need to think differently about concurrent code.

52. Part 3: Infrastructure Concurrency

Concurrent applications running concurrently on distributed computers.

  • Client Application
  • Backend Application
  • Databases
  • Host and Network Infrastructure

53. Student Context

  • Simple architectures:

    • Monolithic
    • Single DB
  • Few concurrent users

54. Professional Context

  • Complex, sometimes messy architectures.

    • Containers
    • Queues
    • Workers
    • Tech Debt
    • Multiple Servers
    • Microservices
    • Multiple datastores
    • Distributed databases
  • Many concurrent users
  • Running longer (more vulnerable to failures, downtime)

55. Example

images/infrastructure-data-race.jpg

56. Review

  • What concurrency is

    • Sequential versus concurrent
    • Comparison with recipes
    • Compared to parallelism

  • How it is implemented

    • Operating System Scheduler and Interrupts
    • Languages and Runtimes

57. Review

  • When, why, and how to use it

    • I/O + UI
    • Similar, independent computations
    • Expressiveness

  • Pitfalls

    • Synchronization
    • Resources Usage
    • Debugging
    • Complexity

58. Review

  • Infrastructure

    • Data Race

59. Conclusion

  • Goals:

    • Give you a rough sense of what concurrency is.
    • Provide context to help you understand what's going on.
    • Practical exposure, to expose some of the trade-offs.
  • Awareness and Sensitivity

  • Remember:

    • Premature optimization is the root of all evil.
    • Tradeoffs and economics (resources, costs, developer time).
    • Learning is continuous…

60. Further Reading

Reference in New Issue View Git Blame Copy Permalink