今日もCrawlerネタ。
「Yahoo-MMCrawler」ってバカ？
何度も何度も同じファイルにクロールしてくる。一体、何だろう？だいらい20秒間隔。
そんなに短い間隔で何度もクロールしたって、ファイル変更してないからっ！っていうか出来ないからっ！
209.191.65.247 - - [18/Nov/2005:14:16:56 +0900] "GET / HTTP/1.0" 200 36539 "-" "Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com)"
209.191.65.247 - - [18/Nov/2005:15:02:54 +0900] "GET / HTTP/1.0" 200 36539 "-" "Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com)"
209.191.65.247 - - [18/Nov/2005:15:03:15 +0900] "GET / HTTP/1.0" 200 36539 "-" "Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com)"
209.191.65.247 - - [18/Nov/2005:15:03:35 +0900] "GET / HTTP/1.0" 200 36539 "-" "Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com)"
209.191.65.247 - - [18/Nov/2005:15:03:55 +0900] "GET / HTTP/1.0" 200 36539 "-" "Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com)"
209.191.65.247 - - [18/Nov/2005:15:04:15 +0900] "GET / HTTP/1.0" 200 36539 "-" "Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com)"
209.191.65.247 - - [18/Nov/2005:15:04:35 +0900] "GET / HTTP/1.0" 200 36539 "-" "Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com)"
209.191.65.247 - - [18/Nov/2005:15:04:55 +0900] "GET / HTTP/1.0" 200 36539 "-" "Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com)"
209.191.65.247 - - [18/Nov/2005:15:05:15 +0900] "GET / HTTP/1.0" 200 36539 "-" "Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com)"
209.191.65.247 - - [18/Nov/2005:15:05:35 +0900] "GET / HTTP/1.0" 200 36539 "-" "Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com)"
209.191.65.247 - - [18/Nov/2005:15:05:55 +0900] "GET / HTTP/1.0" 200 36539 "-" "Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com)"
209.191.65.247 - - [18/Nov/2005:15:06:26 +0900] "GET / HTTP/1.0" 200 36539 "-" "Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com)"
209.191.65.247 - - [18/Nov/2005:15:06:35 +0900] "GET / HTTP/1.0" 200 36539 "-" "Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com)"
209.191.65.247 - - [18/Nov/2005:15:06:55 +0900] "GET / HTTP/1.0" 200 36539 "-" "Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com)"
209.191.65.247 - - [18/Nov/2005:15:09:07 +0900] "GET / HTTP/1.0" 200 36539 "-" "Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com)"
209.191.65.247 - - [18/Nov/2005:15:09:11 +0900] "GET / HTTP/1.0" 200 36539 "-" "Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com)"
209.191.65.247 - - [18/Nov/2005:15:09:31 +0900] "GET / HTTP/1.0" 200 36539 "-" "Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com)"
209.191.65.247 - - [18/Nov/2005:15:09:51 +0900] "GET / HTTP/1.0" 200 36539 "-" "Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com)"
209.191.65.247 - - [18/Nov/2005:15:10:11 +0900] "GET / HTTP/1.0" 200 36539 "-" "Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com)"
209.191.65.247 - - [18/Nov/2005:15:10:31 +0900] "GET / HTTP/1.0" 200 36539 "-" "Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com)"
209.191.65.247 - - [18/Nov/2005:15:10:51 +0900] "GET / HTTP/1.0" 200 36539 "-" "Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com)"
209.191.65.247 - - [18/Nov/2005:15:11:11 +0900] "GET / HTTP/1.0" 200 36539 "-" "Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com)"
209.191.65.247 - - [18/Nov/2005:15:11:33 +0900] "GET / HTTP/1.0" 200 36539 "-" "Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com)"
209.191.65.247 - - [18/Nov/2005:15:11:51 +0900] "GET / HTTP/1.0" 200 36539 "-" "Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com)"
209.191.65.247 - - [18/Nov/2005:15:12:11 +0900] "GET / HTTP/1.0" 200 36539 "-" "Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com)"
209.191.65.247 - - [18/Nov/2005:15:12:31 +0900] "GET / HTTP/1.0" 200 36539 "-" "Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com)"
209.191.65.247 - - [18/Nov/2005:15:13:59 +0900] "GET / HTTP/1.0" 200 36539 "-" "Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com)"
209.191.65.247 - - [18/Nov/2005:15:14:03 +0900] "GET / HTTP/1.0" 200 36539 "-" "Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com)"
コメントする