Sunday, December 13, 2009

Parallel Python is Also Cool

Parallel Python (http://www.parallelpython.com/) is a lightweight library for providing task-level parallelism in Python. It supports both SMP systems as well as clusters. Basically, to use it, you schedule a function for execution with some parameters and context and the scheduler farms out the work to Parallel Python "servers" running on other machines. Most of the annoying details of parallel programming (scheduling, etc.) are taken care of by the library.

Used it today to run a particularly long-running Python script on 27 different processor cores.

No comments:

Post a Comment

Followers