Web Crawling

The Internet is a significant intelligence resource for organizations, and critical in helping them to gather intelligence on targets and threats. The Internet, however, is vast and constantly changing, with new websites springing up every day that have links to gangs, terrorism and domestic extremists. To monitor websites and be informed as soon as a piece of information changes is exceedingly labor intensive.   

So how does the intelligence expert cope?  To manually check websites of interest is arduous and time consuming, and a regular search is needed to find new websites which have just appeared.  

What if you could specify exactly the type of information you were looking for and then simply set up your own web crawler to search for this information regularly then archive it directly to your intelligence management system? Memex is designed to do exactly that.

A valuable add-on utility to your solution, Memex web crawling capability allows you to set up your own web crawlers which meet your precise web search requirements. To the end user, Memex offers a toolbar which integrates into Microsoft® Internet Explorer®, and allows you to index pages directly from your browser.  Behind the toolbar is a server process which allows the web crawlers to be run manually, or automatically in the background, finding critical intelligence from the internet, and allowing you to get on with your core intelligence function.  

Memex can search websites for the intelligence you need, indexes it, and then archives it for you at the appropriate place in your intelligence management system, making it available for later analysis.