I paused it a lot as I didn't really get involved much but finally managed to complete it.
Key Lessons
- Go lang examples for threading, locking, RPC, Typesafe and memory safe, Garbage Collected
- Threads - Tools to manage concurrency in programs
- Stacks are within address space of the program
- I/O Concurrency - Overlapping of progress of different activities wait ing / executing
- Parallelism - Parallelize CPU / IO cycles / routines
- Process is a single program / single address space. Inside process there are multiple threads
- Process -> memory area -> routines sit inside the process
- Process implemented by the operating system
- Thread challenges - Sharing data
- Mutex / Locks for shared data
- Data Access - Managing Locks / Deadlocks / Starvation / Blocking
- Channels (Go Lang) - Send data between threads
- WaitGroup, Sync.Cond
- Webcrawlers design for parallel processing using threads
- Handling concurrency / multiple parallel threads / optimum network capacity utilization
- Remember doing SSIS ETL parallel tasks for Data pull
Crawler
Multi-Threaded Crawler in Python
Happy Learning!!!
No comments:
Post a Comment