User-agent: gsa-test-crawler Allow: / User-agent: gsa-crawler Allow: / #Exclude all others User-agent: * Disallow: /