Robots.txt Simulator

Simplified model: longest matching Allow/Disallow prefix wins within matching User-agent groups. Real crawlers may differ.