Skip to content
This repository was archived by the owner on Jan 23, 2024. It is now read-only.
This repository was archived by the owner on Jan 23, 2024. It is now read-only.

fiber.Pool runs tasks in serial in local mode #57

@coldfire913

Description

@coldfire913

Hi,

I am trying fiber v0.2.1 on local machine. the experiment runs the same task on 4 processes. the task takes approx. 68 secs. while
multiprocessing lib works as expected, fiber looks like to be running in serial.

.fiberconfig
[default]
log_level=info
log_file=stdout
backend=local

def process(mplib, problem, num_tasks):
pool = mplib.Pool(processes=4)
futs = [
pool.apply_async(simple_solve, kwds={"problem": problem})
for _ in range(num_tasks)
]
return [fut.get() for fut in futs]

res, elapsed = timeit(process, mp, problem, num_tasks)
res2, elapsed2 = timeit(process, fiber, problem, num_tasks)

print("multiprocessing takes {} secs, results = {}".format(elapsed, res))
print("fiber takes {} secs, results = {}".format(elapsed2, res2))

multiprocessing takes 92.11180663108826 secs, results = [{'pid': 27242, }, {'pid': 27240, }, {'pid': 27241, }, {'pid': 27243, }]
fiber takes 255.9156596660614 secs, results = [{'pid': 27265, }, {'pid': 27265, }, {'pid': 27265, }, {'pid': 27265, }]

fiber is always using the same process pid 27265 either push or pull queue, while the multiprocessing lib is distributing the works to 4 worker processes.
but fiber's parzen_estimation example is working as expected on local machine.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions