Search Appliance SBE
Syntax: one or more URLs, one per line
Here you may specify URLs for individual pages to include in the index. These pages are fetched and stored in the database like others but the hyperlinks on them are not followed during a walk.
If you change this and click "Update" instead of "GO" the added pages
will be fetched immediately and added to the existing database.
Pages removed from the list will not be removed from the database
until the next rewalk.
Note that since this setting is intended for "one-off" individual URLs, robots.txt will not be fetched for these pages.