DERC Framework (Data Extraction, Reporting and Crawling Framework).

A sophisticated automated data intelligence framework that can discretely crawl, analyze, process and integrate target data with any customer system. The system was built using sophisticated algorithms to gather data and appear like any other traditional web data on the Internet. The flexibility of the framework allows it to be customized to suit any particular need including: data submission, mining, data aggregation, change detection alerts (fax/e-mail/text message/phone). The framework can also be used in a “stealth” mode, to mask the framework’s originating ip and geo-location.

The system’s “personal action response” framework allows the system to be trained to respond to different events. For instance, it has been used to monitor a search database and automatically “submit” a form when certain results are detected. It was trained to recover on failure of the remote system database, so that it would always return the correct results for the client.

The system’s flexibility allows it to be trained for any type of trigger points, including submitting data/documents securely through forms or other forms of electronic gateways (fax, ssh, ftp, text,sms,ssl,etc). In essence the framework can be used for whatever the client’s business requirements.

The system has been refined over several years with project ranging from personalized action response monitoring to full Department of Transportation safety monitoring system for DOT Truck companies.

The framework can be hosted on any Linux/Windows platform and can be easily be hosted within any virtualized environment (VMWare Server/ESXi/Virtualbox/etc).