Plan and configure crawl rules