How to Do Pipeline Forensics, DFIR, or Just Find Files Quickly With Recon
A quick introduction to the open-source tool
Recon is an open-source tool built in Rust 🦀, with great performance and ergonomics for finding files.
You might want to:
- Run some validations or checks in your pipeline — for example, not producing binary files while building Javascript,
- Or you might want to run a digital forensics task on a specific host looking for a specific malware.
- Or, you might want to have a swiss-army knife for locating files using SQL.
Check out the Github repo here.
Key Features
- Query with SQL over files — find files using the full power of SQL queries
- Find content with digests — use SHA256/512, md5, crc32 for duplicates, and other matchers for nontrivial matches to locate artifacts on hosts
- Find malware or binaries with YARA — use YARA rules for matching against binary files efficiently
- Finetune your search runtime — you can choose only the processing you need in order to cover more files quickly
- Build your own scripts — pipe
reconresults to your own scripts with--xargs - Export — use
--csvor--json, or uploadrecon.dbto your own servers for analysis
Getting Recon
Recon is free and open source. For macOS:
$ brew tap rusty-ferris-club/tap && brew install reconOtherwise, grab a release from releases and run recon --help:
$ recon --help
SQL over files with security processing and tests
Usage: recon [OPTIONS]
Options:
-c, --config <CONFIG_FILE> Point to a configuration
-r, --root <ROOT> Target folder to scan
-q, --query <SQL> Query with SQL
-f, --file <DB_FILE> Use a specific DB file (file or :memory: for in memory) [default: recon.db]
-d, --delete Clear data: delete existing cache database before running
-u, --update Always walk files and update DB before query. Leave off to run query on existing recon.db.
-a, --all Walk all files (dont consider .gitignore)
--no-progress Don't display progress bars
-m, --inmem Don't cache index to disk, run in-memory only
--xargs Output as xargs formatted list
--json Output as JSON
--csv Output as CSV
--no-style Output as a table with no styles
--fail-some Exit code failure if *some* files are found
--fail-none Exit code failure if *no* files are found
--verbose Show logs
-h, --help Print help information
-V, --version Print version informationA better ‘find’
Do you remember all of the arguments to find? Me neither. Recon simplifies finding files by letting you use SQL:
$ recon -q "select path,mode,uid from files where ext='rs'"
┌────────────────────────────┬────────────┬─────┐
│ path │ mode │ uid │
├────────────────────────────┼────────────┼─────┤
│ ./xtask/src/main.rs │ -rw-rw-r-- │ 501 │
├────────────────────────────┼────────────┼─────┤
│ ./recon/tests/cli_tests.rs │ -rw-r--r-- │ 501 │
├────────────────────────────┼────────────┼─────┤
│ ./recon/src/os.rs │ -rw-r--r-- │ 501 │
├────────────────────────────┼────────────┼─────┤
│ ./recon/src/out.rs │ -rw-r--r-- │ 501 │
├────────────────────────────┼────────────┼─────┤
│ ./recon/src/bin/recon.rs │ -rw-r--r-- │ 501 │
├────────────────────────────┼────────────┼─────┤
│ ./recon/src/config.rs │ -rw-r--r-- │ 501 │
├────────────────────────────┼────────────┼─────┤
│ ./recon/src/lib.rs │ -rw-r--r-- │ 501 │
├────────────────────────────┼────────────┼─────┤
│ ./recon/src/data.rs │ -rw-r--r-- │ 501 │
├────────────────────────────┼────────────┼─────┤
│ ./recon/src/processing.rs │ -rw-r--r-- │ 501 │
├────────────────────────────┼────────────┼─────┤
│ ./recon/src/workflow.rs │ -rw-r--r-- │ 501 │
├────────────────────────────┼────────────┼─────┤
│ ./recon/src/matching.rs │ -rw-r--r-- │ 501 │
└────────────────────────────┴────────────┴─────┘
11 of 233 files in 5.565875msHow about doing something with the results with the usual xargs combo?
$ recon -q "select path from files where ext='rs'" --xargs | xargs echo
./xtask/src/main.rs ./recon/tests/cli_tests.rs ./recon/src/os.rs ./recon/src/out.rs ./recon/src/bin/recon.rs ./recon/src/config.rs ./recon/src/lib.rs ./recon/src/data.rs ./recon/src/processing.rs ./recon/src/workflow.rs ./recon/src/matching.rsA powerful forensics tool
Recon can also be useful for security specialists trying to locate vulnerable binaries, or malware, and any other DFIR task at hand.
You can configure processors and matchers in case you need some compute-heavy data to be available for query or matching such as:
- Digests or checksums (md5, SHAxxx, crc32)
- Path, content matching
- YARA rules
- For more, see https://github.com/rusty-ferris-club/recon/tree/master/examples
Let’s add binary detection. Create a configuration file called config.yaml:
source:
computed_fields:
is_binary: trueDid you know? The common way to detect if a file is binary, is to read a chunk of data from it (usually 1k bytes) and search for any non-text bytes in it. Largely: there’s no built-in OS metadata “flag” that indicates if a file is a binary file.
We can use -m to avoid saving any cache to disk.
$ recon -m -c config.yaml -q 'select path,is_binary,mode from files limit 6'
┌───────────────┬───────────┬────────────┐
│ path │ is_binary │ mode │
├───────────────┼───────────┼────────────┤
│ ./os.rs │ 0 │ -rw-r--r-- │
├───────────────┼───────────┼────────────┤
│ ./out.rs │ 0 │ -rw-r--r-- │
├───────────────┼───────────┼────────────┤
│ ./bin/main.rs │ 0 │ -rw-r--r-- │
├───────────────┼───────────┼────────────┤
│ ./config.rs │ 0 │ -rw-r--r-- │
├───────────────┼───────────┼────────────┤
│ ./lib.rs │ 0 │ -rw-r--r-- │
├───────────────┼───────────┼────────────┤
│ ./recon.db │ 1 │ -rw-r--r-- │
└───────────────┴───────────┴────────────┘
6 files in 145.81675msExploring more
We have fully configured examples to explore and start from (open a PR to add your own!):
all-processors.yaml — Turn on all processors by default
custom-walking.yaml — Compute fields only for part of the walked directory tree
file-classes.yaml — Configure and classify your own file classes
find-log4shell.yaml — An example for finding the log4shell vulnerability using known digests
using-yara.yaml — Using a simple YARA ruleset for matching
Have a use case? want to contribute back? feel free to open a PR
