Site Update Notification

The Site Update Notification project aims to deliver a mechanism for web masters to automatically publish information about changes to their website (changes in content and in presentation and layout) directly to those interested in keeping up-to-date with their publications. This could be Search Engines, Page Monitors, Bookmark Managers, Offline Browsers, Website Mirroring, Web Archives or others.

Architecture

This is an alternative to the traditional crawling of sites that download pages and following hyperlinks to related pages (crawling the web) and distributes the burden from a centralised (albeit distributed) web spider and increases efficiency by reducing bandwidth requirements across the internet.

Read more...

Keywords

crawler, spider, web crawler scalability, optimization, change notification, site update notification

Project Goals

The Site Update Notification System seeks to ...

  • address the problem with crawler scalability
  • reduce the crawler bandwidth and index load
  • reduce the time that a web site’s change takes to be available in a search engine’s index
  • address the problem crawlers currently face in accessing the Deep Web
  • provide an infrastructure for agent data aggregation and delivery to subscribers that reduces the bandwidth of individual web sites

Read more...

Site Map


Home

Web Crawling

     Deep Web

     sunaweb spider

Solution

     Agents

         SUNA's

     Products

         Web Crawlers

         Web Masters

         Hosting Servicing Companies