Navigation X
ALERT
Click here to register with a few steps and explore all our cool stuff we have to offer!



 1881

Threading + efficient data handling system (PYTHON)

by dylan6 - 29 March, 2023 - 11:36 PM
This post is by a banned member (dylan6) - Unhide
dylan6  
Infinity
167
Posts
25
Threads
3 Years of service
Hello, I am looking for a developer to complete the following task.

Below is the full detail of the script and what it should do.
The script is a proof of concept that you will be making, you will only be tasked to print certain data, and prove it can handle millions lines of data without being slow.

There should be this list of files for data:
Links.txt - Just a simple link variable to be able to be used in the code
SentLinks.txt - Every Links.txt visited from above will be put here, so that the link is not revisited from other runs, or from the same run with a different thread
Accounts.txt - This is another variable to be used in the code
Proxies.txt - Another variable to be used in the code

So a code example can be written below by you with an efficient threading system that supports million lines of data and doesnt repeat:
 ```py
def link_visit():
    print(account + ' Visited ' + link + ' With ' + proxy)
```
Thats all I want the code to be able to do, is print that variable information, but all work together in threads. The threading should be fast and efficient.


For example links would look like
```
link1
link2
link3
... up to millions of links
```

As well there will be a sentlinks.txt
This is where the optimization needs to be coded. 
Normally, when you append lines on python- it freezes the program (because it is reading every line to make sure it is not in links.txt) and then executes the code.

I want a very fast way to organize sentlinks.txt from links.txt

The sentlinks are links that basically loop to be removed from links.txt. To ensure that those links aren't repeated.

the script should make the accounts visit 1 link per run.

So I run the program with 100 accounts, it should select 100 non visited links to visit with those 100 accounts for that run.

Then I can I run the program again, with 200 accounts for example, and then 200 accounts will visit 200 links that still have never been visited before.

I plan to run it like this, so a lot of data will be building up in links and sentlinks.txt. So I should be able to handle millions without slowing down anything

I am open to hear your ideas on how to make this efficient, I just need a proof of concept script basically with the threading do

Requirements:
- Make sure each account inputted is used one time
- Choose 1 link for the account to visit in that run
- The link must not be repeated ever
- Links additionally stored in sentlinks.txt should support millions and not slow down the program at all. For example, the program shouldn't freeze when I start it. 
- Python
- Make sure to follow every other step mentioned previously while explaining the post. Please ask questions before starting if you aren't sure. 

I am open to hear ideas on how to solve this issue in python, as I really can't have any repeats, and I can't be waiting forever to start the program even when millions of lines are stored.

A realistic number of lines for the files will be around 10mil-50mil.
Both in links.txt and sentlinks.txt

Proxies will contain around 100k max
Accounts will contain around 300k max

Proxies and accounts do not need to be related, the proxies can be randomly assigned.

PM me on cracked and leave your discord. I will check my PM's often
If you enjoyed my content I shared, consider leaving a like and +repping to encourage me to continue providing contributions

Note: I do not provide support on site regarding my shop, please contact on telegram and be sure to read our terms
https://discordu.sellix.io
https://t.me/discordustock
 
This post is by a banned member (israinsolutions) - Unhide
3.871
Posts
14
Threads
4 Years of service
#2
Hey, I can help you with that!

DM me on telegram: https://t.me/israinsolutions
MY NEW DISCORD THAT IS MINE IS "israinsolutions" (ID: 685020161870135305) & MY ONLY TELEGRAM: https://t.me/israinsolutions
MAKE SURE YOU DOUBLE CHECK IT, ALSO IF ANYONE CONTACTS YOU SAYING I AM SELLING ANYTHING REMEMBER IT IS NOT ME
IMPOSTERS ALERT | Discord/Telegram/Skype In SignatureAlways Confirm On-Site

[Image: QH0dsPY.gif]
This post is by a banned member (dylan6) - Unhide
dylan6  
Infinity
167
Posts
25
Threads
3 Years of service
Bumped #3
This is a bump
This post is by a banned member (dylan6) - Unhide
dylan6  
Infinity
167
Posts
25
Threads
3 Years of service
#4
(This post was last modified: 01 April, 2023 - 11:27 PM by dylan6. Edited 1 time in total.)
Completed for $20, fuck off all the other people over charging for shit
If you enjoyed my content I shared, consider leaving a like and +repping to encourage me to continue providing contributions

Note: I do not provide support on site regarding my shop, please contact on telegram and be sure to read our terms
https://discordu.sellix.io
https://t.me/discordustock
 
This post is by a banned member (Twez) - Unhide
Twez  
Supreme
1.103
Posts
98
Threads
2 Years of service
#5
(29 March, 2023 - 11:36 PM)dylan6 Wrote: Show More
Hello, I am looking for a developer to complete the following task.

Below is the full detail of the script and what it should do.
The script is a proof of concept that you will be making, you will only be tasked to print certain data, and prove it can handle millions lines of data without being slow.

There should be this list of files for data:
Links.txt - Just a simple link variable to be able to be used in the code
SentLinks.txt - Every Links.txt visited from above will be put here, so that the link is not revisited from other runs, or from the same run with a different thread
Accounts.txt - This is another variable to be used in the code
Proxies.txt - Another variable to be used in the code

So a code example can be written below by you with an efficient threading system that supports million lines of data and doesnt repeat:
 ```py
def link_visit():
    print(account + ' Visited ' + link + ' With ' + proxy)
```
Thats all I want the code to be able to do, is print that variable information, but all work together in threads. The threading should be fast and efficient.


For example links would look like
```
link1
link2
link3
... up to millions of links
```

As well there will be a sentlinks.txt
This is where the optimization needs to be coded. 
Normally, when you append lines on python- it freezes the program (because it is reading every line to make sure it is not in links.txt) and then executes the code.

I want a very fast way to organize sentlinks.txt from links.txt

The sentlinks are links that basically loop to be removed from links.txt. To ensure that those links aren't repeated.

the script should make the accounts visit 1 link per run.

So I run the program with 100 accounts, it should select 100 non visited links to visit with those 100 accounts for that run.

Then I can I run the program again, with 200 accounts for example, and then 200 accounts will visit 200 links that still have never been visited before.

I plan to run it like this, so a lot of data will be building up in links and sentlinks.txt. So I should be able to handle millions without slowing down anything

I am open to hear your ideas on how to make this efficient, I just need a proof of concept script basically with the threading do

Requirements:
- Make sure each account inputted is used one time
- Choose 1 link for the account to visit in that run
- The link must not be repeated ever
- Links additionally stored in sentlinks.txt should support millions and not slow down the program at all. For example, the program shouldn't freeze when I start it. 
- Python
- Make sure to follow every other step mentioned previously while explaining the post. Please ask questions before starting if you aren't sure. 

I am open to hear ideas on how to solve this issue in python, as I really can't have any repeats, and I can't be waiting forever to start the program even when millions of lines are stored.

A realistic number of lines for the files will be around 10mil-50mil.
Both in links.txt and sentlinks.txt

Proxies will contain around 100k max
Accounts will contain around 300k max

Proxies and accounts do not need to be related, the proxies can be randomly assigned.

PM me on cracked and leave your discord. I will check my PM's often

GLWSHeart
[Image: HeLAyHj.gif]
[Image: yy59jR8.gif]
This post is by a banned member (words) - Unhide
words  
Supreme
1.480
Posts
324
Threads
4 Years of service
#6
(This post was last modified: 03 April, 2023 - 06:04 PM by words. Edited 2 times in total.)
Hey, why do you wanna use python multithreading, it's so trash tbfh i know python & the GIL makes it almost crash your whole cpu lol

edit : also it might not be efficient to load millions of lines in py, this would slow down & be heavy in memory ?

more suitable way would be golang, this would handle everything better & make it even more faster with LOT less cpu usage & mem usage, get yo hands on a golang dev for this if u want a very efficient code ;)
I do not sell anything. I do not have a public Discord / Telegram. I will never ask anyone for money / refuse any deal via middleman. I will never deny to confirm my identity via PM before any deal. Stay safe, I have impersonators.
(-.-)Zzz...

https://youtu.be/EDVrTYW2l84

 
This post is by a banned member (dylan6) - Unhide
dylan6  
Infinity
167
Posts
25
Threads
3 Years of service
#7
(03 April, 2023 - 06:01 PM)seinen Wrote: Show More
Hey, why do you wanna use python multithreading, it's so trash tbfh i know python & the GIL makes it almost crash your whole cpu lol

edit : also it might not be efficient to load millions of lines in py, this would slow down & be heavy in memory ?

more suitable way would be golang, this would handle everything better & make it even more faster with LOT less cpu usage & mem usage, get yo hands on a golang dev for this if u want a very efficient code ;)

already got this completed, and you are just so wrong here.
you just dont update the list instantly and instead make it a queue.

cpu usage at 500 threads currently for me takes less than 3% cpu of my ryzen 7 5800x. no clue what u r talking about. 
we are just printing here explained in the thread..
If you enjoyed my content I shared, consider leaving a like and +repping to encourage me to continue providing contributions

Note: I do not provide support on site regarding my shop, please contact on telegram and be sure to read our terms
https://discordu.sellix.io
https://t.me/discordustock
 
This post is by a banned member (words) - Unhide
words  
Supreme
1.480
Posts
324
Threads
4 Years of service
#8
(03 April, 2023 - 11:06 PM)dylan6 Wrote: Show More
(03 April, 2023 - 06:01 PM)seinen Wrote: Show More
Hey, why do you wanna use python multithreading, it's so trash tbfh i know python & the GIL makes it almost crash your whole cpu lol

edit : also it might not be efficient to load millions of lines in py, this would slow down & be heavy in memory ?

more suitable way would be golang, this would handle everything better & make it even more faster with LOT less cpu usage & mem usage, get yo hands on a golang dev for this if u want a very efficient code ;)

already got this completed, and you are just so wrong here.
you just dont update the list instantly and instead make it a queue.

cpu usage at 500 threads currently for me takes less than 3% cpu of my ryzen 7 5800x. no clue what u r talking about. 
we are just printing here explained in the thread..

okay so queue work line per line ? thought shit needed to lead instantly mb

is it using multithreading or multiprocessing ?
I do not sell anything. I do not have a public Discord / Telegram. I will never ask anyone for money / refuse any deal via middleman. I will never deny to confirm my identity via PM before any deal. Stay safe, I have impersonators.
(-.-)Zzz...

https://youtu.be/EDVrTYW2l84

 

Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
or
Sign in
Already have an account? Sign in here.


Forum Jump:


Users browsing this thread: 1 Guest(s)