The SearchmetricsBot crawls all pages within the specified project. This means, if only a domain is specified (without www) in the project area, all subdomains can be crawled. If a subdomain is specified in the project area, only pages within this subdomain can be crawled. If a directory is specified, only the documents within the directory are crawled.
