Navigation X
ALERT
Click here to register with a few steps and explore all our cool stuff we have to offer!



 1459

Need help at ThreadingPool!

by Pikapyy - 26 January, 2022 - 08:28 PM
This post is by a banned member (Pikapyy) - Unhide
Pikapyy  
Supreme
375
Posts
14
Threads
4 Years of service
#1
Hello, I making a LoL keker and i cant integrat eThreading Pool to my src, can anyone help me on Threading Pool?
DISCORD: pika?#1888
TELEGRAM: @Pikapyy
Thanks in advance. PepeGlad
[Image: standard.gif]
Always confirm via PM before having a deal.
This post is by a banned member (Pikapyy) - Unhide
Pikapyy  
Supreme
375
Posts
14
Threads
4 Years of service
#2
This is a bump
[Image: standard.gif]
Always confirm via PM before having a deal.
This post is by a banned member (UberFuck) - Unhide
UberFuck  
Godlike
1.557
Posts
375
Threads
5 Years of service
#3
I usually rely on the following for most of my stuff:
Code:
import contextlib
import joblib

@contextlib.contextmanager
def tqdm_joblib(tqdm_object):
    """Context manager to patch joblib to report into tqdm progress bar given as argument"""
    class TqdmBatchCompletionCallback(joblib.parallel.BatchCompletionCallBack):
        def __init__(self, *args, **kwargs):
            super().__init__(*args, **kwargs)

        def __call__(self, *args, **kwargs):
            tqdm_object.update(n=self.batch_size)
            return super().__call__(*args, **kwargs)

    old_batch_callback = joblib.parallel.BatchCompletionCallBack
    joblib.parallel.BatchCompletionCallBack = TqdmBatchCompletionCallback
    try:
        yield tqdm_object
    finally:
        joblib.parallel.BatchCompletionCallBack = old_batch_callback
        tqdm_object.close()


So instead of something like this:
Code:
class Checker:
    def __init__(self) -> None:
        DoSomething()

    def Check(self, combo):
        print(f'Checking {combo}')

for combo in combos:
    Checker.Check(combo)

You would use this:
Code:
from joblib import Parallel, delayed
from tqdm import tqdm

class Checker:
    def __init__(self) -> None:
        DoSomething()

    def Check(self, combo):
        tqdm.write(f'Checking {combo}')

with tqdm_joblib(tqdm(desc="Checking", total=len(combos), unit='combo', colour='GREEN', leave=False)):
    Parallel(n_jobs=20, require='sharedmem')(delayed(Checker.Check)(combo) for combo in combos)

It gives you a basic progress bar and executes whatever function you are needing in parallel.  The only thing changed in your code would be instead of using print() for console output, you should use tqdm.write(), otherwise it messes with the progress bar.

Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
or
Sign in
Already have an account? Sign in here.


Forum Jump:


Users browsing this thread: 1 Guest(s)