- Регистрация
- 26.10.2011
- Сообщения
- 512
- Благодарностей
- 455
- Баллы
- 63
Hi Guys
I had many questions about how to scrape something these days, so i have builded YAHOO ANSWERS SCRAPER and im willing to share it with you. It goes to yahoo answers and search for questions/answers using keywords from list which must be placed in same folder as template and named "keywords.txt" (It will pick only best answers). Im sure this will come handy to many of you and so here you go
There is "SET NUMBER OF PAGES TO SCRAPE" branch on right side of template, it is set to {-FieldData.FieldData-|-●5●6●7●6●4●6●-|-≡1≡8≡0≡0≡6≡7≡2≡1≡7≡0≡-}>10 ...this number 10 means it will scrape 10 pages of search results (100 results), if you wish to scrape more or less pages, you just need to change that number to what ever you like.
This template is made to be as example of "how to", inside you can see few things like:
1. How to make dynamic file name
2. How to use regex on regex
3. How to replace string
4. How to check if file exists
5. How to loop back
6. How to use regex-replace
PS: I hope you learn something from it and give something back to community. In case you dont like sharing but like using this template, please be so nice and press "thank you" button as sign of gratitude!
Here is the template:
Посмотреть вложение YahooAnswersScraper_Custom.xml
I had many questions about how to scrape something these days, so i have builded YAHOO ANSWERS SCRAPER and im willing to share it with you. It goes to yahoo answers and search for questions/answers using keywords from list which must be placed in same folder as template and named "keywords.txt" (It will pick only best answers). Im sure this will come handy to many of you and so here you go
There is "SET NUMBER OF PAGES TO SCRAPE" branch on right side of template, it is set to {-FieldData.FieldData-|-●5●6●7●6●4●6●-|-≡1≡8≡0≡0≡6≡7≡2≡1≡7≡0≡-}>10 ...this number 10 means it will scrape 10 pages of search results (100 results), if you wish to scrape more or less pages, you just need to change that number to what ever you like.
This template is made to be as example of "how to", inside you can see few things like:
1. How to make dynamic file name
2. How to use regex on regex
3. How to replace string
4. How to check if file exists
5. How to loop back
6. How to use regex-replace
PS: I hope you learn something from it and give something back to community. In case you dont like sharing but like using this template, please be so nice and press "thank you" button as sign of gratitude!
Here is the template:
Посмотреть вложение YahooAnswersScraper_Custom.xml
