# This file is needed at the root of each site to avoid a 404 situation. WWW Robots are programs that traverse many # pages in the World Wide Web. They look for robots.txt files to define which parts of the server should not be accessed.