ZennoLab

Automate everything

User Tools

Site Tools


Sidebar

Translations of this page:

en:project-optimization

Efficient usage of PC resources

ZennoPoster - multithreaded program. Each thread handles a lot of information and requires significant amount of PC's resources. This article will help you to configure software for your PC to use it's resources as efficiently as possible

User's time

This is the most important resource. CPU is cheap, RAM - even cheaper. The only thing that really matters - your personal time.
The main purpose of Zennoposter is to save this resource.

RAM

Each thread in ZennoPoster is a separate browser with it's own cookies (about 13 types of them), cache, proxies, user agents and all other data that is necessary for a full emulation of real user.

Due to accommodation of multiple threads in a single process, each thread requires three times less RAM compared to previous versions of ZennoPoster and other competitive programs. You have to consider several factors in order to calculate apox. amount of memory that is needed for properly work of software:

  1. Site that you are working with. All sites are very different, and each of them require totally different amount of RAM. So, it's possible that automation of site #1 in 100 threads requires the same amount of memory as automation of site #2 in 20 threads on the same machine. Try to disable Flash, pictures, and other resources, if they are not needed to operate with site.

  2. Files. To speed up the process files are stored in memory (otherwise there would be problems with access to the hard drive.) If you use different copy of the file for each thread (it occurs when you use lists and tables without syncing with a file (“save changes to file” is not checked in settings)), each one of them requires additional memory to store the file.
    Please note that tables need much more RAM than the size of original file on your hard disk. Lists use less RAM than tables, however, memory also increases. For example, a table which is based on a 10 Mb CSV file, without synchronization with the file may take up to 5 GB RAM for 100 threads. Avoid usage of lists and tables without syncing them with files.

  3. See paragraph (2) in “Processor” chapter.

  4. Search by picture. This feature use a lot of memory. But very useful :)

Processor

Each thread in ZennoPoster builds DOM model, renders images, execute JS, handles events, animations, etc. All of this require CPU resources. Also, the processor is loaded with periodic reset of instances. \\There are several ways to minimize usage of processor:

  1. Disabling unnecessary processing elements, flash, pictures, scripts, and all that is possible. Please note that if a site has good antispam protection, this could affect ban of accounts. For example, registration will take place, but the new account will cause suspicion and in the future may be banned.

  2. In the settings you can set number of threads per base. By increasing this parameter, you save memory and CPU a little, but additional delays while simulating mouse and keyboard strokes will be added.

  3. IntelliSearch and emulation. IntelliSearch increases usage of CPU (2-3 times), but it helps to quickly create projects. To use it or not, its up to you. Mouse and keyboard emulation increases usage of processor and has the effect described in (2). But sometimes without it you just can not create an account. Also usage of emulation during the registration process lowers chances to get ban for your accounts.

  4. Captcha recognition using modules from CapMonster also require CPU resources.

  5. Search by picture use significant amount of CPU resources too. But is still very useful feature :)

Execution time

Pay attention to this parameter. During the captcha recognition via external anticaptcha services project holds for about 15 seconds (until the service workers recognize your captcha).
During this time processor's work for this thread close to 0.
Because of this (if the project has this action) the total number of threads that can be executed on this machine simultaneously increases. Ie you can run project #1, which opens web pages 1 by 1 without captchas recognition (normal parser, for example) in 30 threads. And it will take all of CPU resources. Or you can run project #2 (accounts creation), which has a captcha recognition action in 150 threads. And it will take all of the RAM.
Similarly situation with the confirmation by email, when a thread is waiting for the letter. In this case, project holds too and your machine get resources for other threads.

Intent channel

It's simple. You open a web page - it is downloaded from the server as like as all images, flash, scripts, etc. that are used in it's source. If you clear cache, all data downloaded continuously each time, if not - then usage of this resource is greatly reduced.
Usually you have to clean the cache. Without this action web sites will realize that you have already visited them, even if you use a proxy.
Internet channel can be wide, however, limit of allowed open connections can reduce program's work.

Open connections

You can google to learn more about it. You can use ProxyChecker to test how many of these connections are supported by your network. It has channel test and online monitoring. \\Thread that use a proxy needs 2 connections. Thread without proxies - up to 32. \\Generally, number of available connections is limited by internet providers, but a lot of modems have limits too: usually available around 300 while dedicated servers support thousands of them (up to 15,000 connections on hetzner.de) \\Note that ProxyChecker uses connections too. 1 connection for 1 thread of ProxyChecker.

As a result,

Each project requires a different amount of resources. One can use 15 MB of memory for 1 thread, other 1 GB. Similar with processor's resources - it all depends on your project.

Development of Zennoposter is all about saving your time, everything else is highly customizable and scalable.

Default number of threads

In ZennoPoster you can set the maximum number of threads that can be run simultaneously. There is a recommended value of threads, based on a rough estimate of the power of your computer.
If you read the previous paragraphs, it is clear to you that the number of threads is highly dependent on the project and should be changed manually for your needs. You can slowly add more threads, until one of your computer's resources will be exhausted.
You need to consider one thing: processor resources can be slightly exceeded, ie if you have it loaded to 100% increasing 10-15% in threads count will no harm. Threads will just wait for a while for free resources.
On the other hand, its best to keep 10-30% of free RAM. Lack of it can cause crashing of ZennoPoster and all other programs on your computer.

"Socket" speed

Our unscrupulous competitors (we will not point the finger :) claim that their program can run at a speed of sockets, if you disable flash, scripts, images, etc.
In fact it is not. Browser will never work with speed of direct http requests. Direct requests operate 50-100 times faster. You can verify this by running our ProxyChecker on a dedicated server in 15,000 threads. We tested it on hetzner.de servers in 15,000 threads and we ran out of the available connections, their network hardware wasn't ready for more, and free CPU resources and RAM was enough to go up to 50000.

en/project-optimization.txt · Last modified: 2015/07/14 15:51 (external edit)