• Article  

      Characterizing crawler behavior from web server access logs 

      Dikaiakos, Marios D.; Stassopoulou, Athena; Papageorgiou, Loizos (2003)
      In this paper, we present a study of crawler behavior based on Web-server access logs. To this end, we use logs from five different academic sites in three countries. Based on these logs, we analyze the activity of different ...
    • Article  

      An investigation of web crawler behavior: Characterization and metrics 

      Dikaiakos, Marios D.; Stassopoulou, Athena; Papageorgiou, Loizos (2005)
      In this paper, we present a characterization study of search-engine crawlers. For the purposes of our work, we use Web-server access logs from five academic sites in three different countries. Based on these logs, we analyze ...