Automate everything

User Tools

Site Tools


Translations of this page:


5. Module training and testing

After completing all previous stages you can create and train your recognition module.

General tips for training

For the start we recommend to select simple core and run quick recognition with large number of synbols (1/3 part from total) for test. Try quickly pass a few dozens of training iterations and check the results. If your have got proper results, adjust parameters and run training again. At this time you can train more powerfull network with higher quality. If all is well, it is possible to train a network with more powerful structure and higher quality. Adjust parameters to improve the results and run training again.

Three types of symbol recognition error

1) Wrong recognition - when symbols exists, but recognized incorrectly. For example, you got “c” instead of “a”.
2) Missed symbol - when symbols exists, but module does not see any symbol. In other words, you got nothing instead of “a”.
3) False recognition - when symbol doesn't exist, for example between two symbols, but module found something there.

Module training

Core settings

Core capacity - than higher core capacity, than higher symbol recognition rate. But, in this case, core performance is slower, and more CPU time is required for captcha processing. We recommend to start with quite easy core, as that may be enough for proper recognition result. When adjusting capacity, you can see core complicity assesment below. This parameter reveals how much CPU time is required for processing core at one point. It's just a number, without measuring units, which helps to compare two cores. If one core has twice higher complicity than another, its performance will be twice slower in the same environment. Core complicity also depends on symbol recognition window size. Thus, adjusting these two settings, you can create recognition core with required recognition rate and processing speed.

Training settings


Training speed - than slower training, than high recognition rate of trained module. But we don't recommend to start with slow and careful training. Try to train your module quickly for the first, as there may be some mistakes at previous stages and you will be able to train your module carefully after correcting them. For simple captcha, quick training may be even enough to achive proper results.

Start parameter - this parameter controls start speed of training. Leave it default. It should be changed only in case you get one of the errors described below.

Centers variation - during training collected symbols are shown to the core, their centers will be in the points you marked. You can increase variation of these points, which may improve recognition or, vice versa, make it worse. In the next paragraph “Module testing” you can find information on when this parameter is useful.

Training for recognizing false symbols - should be increased if you get symbol recognition error (3) - false recognition error occurs much more often than other errors. We recommend to start training with default value: 10.

Training symbols portion - core training will be performed on collected symbols, but not all, some of them will be used for core testing (first graph). Testing is very important as it allows to understand how successful is training process and which settings needed to be adjusted for better results. Core testing requires at least 30-50 symbols, but not much more. It's better to start with large number of symbols for testing and than after adjusting parameters repeat training with minimum number of symbols.

Recognition settings

Recognition threshold - this parameter takes value between 0 - no symbols in highlighted area, and 1 - symbol certainly exists in highlighted area. But we need to know exactly if there is a symbol, and recognition threshold will help us with it. All values that higher than this threshold mean that there is a symbol, and less - there is no. Below we will describe how to avoid some training problems adjusting this parameter.

Min distance between symbol centers - very userful parameter, which prevents searching for any other symbol in area of already recognized symbol, allowing to minimize recognition errors (3). Open captcha image with Paint, scale it according to your filters and measure min distance (in pixels) between two symbols. Guess such distance in all captcha images and set minimal possible as value for this parameters. If you are mistaken, you will be able to correct it during module testing.

Captcha images for testing - recognition test will being constantly performed during module training, to estimate recognition rate of the module. If you have many images - tests will be long, but with proper estimation, if few images - tests will be performed quickly, but with not proper estimation. After training module a couple of times, you will understand what value is better to set.

Frequency of testing - defines how often to perform testing. By default, module test is performed on every training cycle. Using this parameter you can define after how many cycles testing will be performed.

Quick recognition - complex captchas (symbols are very close to each other, in one line without spaces, heavily distorted, noisy, with many different kinds symbols) require to set large scale, high complicity for module core, large mass centers variation, etc. Because of this, module training may take too much time. This parameter enables some optimization and allows to speed up recognition tests in 5-10 times. But it may reduce recognition rate. You can try enabling this optimization at module testing. To correct recognition rate you may have to adjust mass centers search settings. We recommend not to enable quick recognition parameter for module training, as you may not achive highest possible recognition result for such captchas.

Training progress

After setting all the parameters, you can run training. Training progress will be displayed in a window with graphs and progress bar which shows current phase of training. There are 3 graphs in the window:

1) The first displays results of testing module on collected symbols:
Green line - correctly recognized symbols
Yellow line - symbols were not found (second recognition error)
Red line - incorrectly recognized symbols (first recognition error)

2) The second displays results of false recognition test:
Green line - right answers (core has not found any symbols)
Yellow line - suspected recognition (core recognized something, but there is no any symbol)
Red line - false recognition (third recognition error)

3) The third displays approximate preliminary recognition rate of a module.

Stopping training

Training process can be stopped anytime, if you think that you won't get better results anymore.

After stopping or completing (300 cycles) training you should select the core which will be used in your module:

1) Latest result - if you used minimum number of captcha images (less than 500) in latest loop and achieved results almost equivalent to best results, we recommend to select core with latest result.

2) Best result - the core with best result achieved during trainig will be selected for module.

3) Use the result of previous training - if you achieved better results with previous settings, you can selet core with previous result for your module.

How training should proceed

If training proceeds sucessfully, the graphs should have following view:

On the first graph (symbols recognition) green line rises to the maximum, and yellow and red go down to 0.

On the second graph (false recognitions) green line is always on top, red line close to 0 and yellow line 5-10 times higher than red line.

On the third graph (captcha recognition rate) - blue line is growing gradually, at start quickly and then slower.

This screenshot was made while training module for simple captcha with right-chosen settings, keep in mind that you may get another view on graphs, start recognition rate may be lower, rise of green line, as well as reduction of yellow and red lines on the first graph may be slower. All this depends on training settings and captcha complicity. The most important is the tendency during training.

Training problems and their solving

The first rule - if something doesn't work during long time and the tips given below don't help, ask your question on users forum in thread for the program. Don't waste your time!
There are several problems that you may encounter, while training your module. They can be diagnosed with help of graphs that display training process:
1) It has been passed 10 cycles already after I started, but green and red lines in the first graph are not rising.
That means that the core is not being trained. Start parameter for the training is too low, increase its value in 10 times and start training again. Repeat increasing until lines on the graph won't start rising.
2) Red line rose above green line and dominates on the first graph.
*Probably, start parameter for the training is too high. Try reducing it in 10 times and training core again. Repeat it until problem will be solved. If you face with problem (1) reducing start parameter, that means that the problem is caused by other things.
*Maybe you have collected not enough symbols for training or applied filters for symbols with incorrect settings. Collect more symbols or re-configure filters.
*Perhaps, you have messed something up when collected symbols and gave them incorrect name. Check out the collection in the filter section (there's a button “show text” pressing which you can check whether symbol on captcha image matches to written text below it).
3) Red line on the first graph is quite far from 0.
Too many recognition errors of type (1).
There may be many reasons for this:
* Not enough symbols collected
* Too complicated captcha
* Too simple network
* Training speed is too high
* Centers variation is too large
* Too many types of symbols (may be if you are trying to solve captcha with chinese characters).
4) The first tesing, which is performed before the first cycle, returns too many Bad results (red line).
There is nothing wrong with this, probably you have set recognition threshold value less than 0,5. Results will be improved already after first cycle.
5) The number of cycles has exceeded 100, but yellow line on the first graph still doesn't drop to zero.
The core doesn't see too many symbols - recognition error (2).
Possible reasons:
*To high recognition threshold. Recommended value for this parameter is 0,5-0,6
*To low start parameter for training. Try increasing it in 3-5 times.
*Not enough symbols for training and they are distorted too much.
*Too complicated network
*Too high training speed.
6) On the second graph red line has grown too high.
The core sees false symbols too often - third recognition error (3). To avoid it increase value of Training for recognizing false symbols parameter.
7) Recognition rate on the third graph is not growing, although other graphs are correct.
Training is going well, but captcha for some reason is not recognized. You need to check what is wrong in test. Complete module training so that the graphs 1 and 2 keeped approximately the same values and run module testing. See what recognition errors occur and read help paragraph for module testing to find out how to fix these recognition errors.

en/addons/capmonster/learning/module-teach.txt · Last modified: 2015/10/15 13:08 by vladz