Back in March 2021 we talked about using Sophos Central Live Discover API to leverage MISP threat intel to search your estate for known IOCs; we hope it has proven useful. This powerful mechanism to search the estate had one drawback: the endpoint needed to be online in order to run a live query to hunt for those IOCs. This could lead to offline endpoints being overlooked when the query is run, which may have seen the IOCs.
With the arrival of Sophos Central XDR API, you now have the capability to search the Data Lake for those same IOCs and it doesn’t matter if that machine hasn’t been switched on for the last few hours. The Data Lake gives you access to up to 30 days of data on all the endpoints you want to query.
Hydration queries periodically run on the endpoint and the resulting data is uploaded to your Data Lake, providing this capability is enabled, providing a swathe of data to mine for known IOCs.
In the same way you trigger queries using Live Discover, the same is true for the Sophos Central XDR API and we have expanded the Sophos Central API Connector to easily run saved queries on all your tenants in one command.
Start hunting
To get started using the IOC Hunter follow the below steps to run your first query.
Step 1: Install Sophos. Central API Connector
First things first, make sure you have installed the latest version of the Sophos Central API Connector. The easiest method to get started is to follow the quick start guide.
Step 2: Create the IOC Hunter query on your tenants
The query itself differs from the Live Discover query due to schema differences and a new query can be found on our team GitHub page to save to your tenants. You can save a custom query by following these instructions.
Create your variable name and type as:
Remember the name you gave your custom query when you saved it as you will need it later when running ioc_hunter.py when setting −−search_input value
Step 3: Set MISP Configuration
Not using MISP? You can skip this step.
If you are not using the AWS Secrets Manager to store your API key, simply define your MISP instance URL in the `misp_config.ini` file under the misp_instance section.
To find both your API key and instance URL, login to your MISP instance and go to: Home > Automation.
If you are utilizing AWS Secrets Manager to store your API key and are unfamiliar with the steps, follow this AWS tutorial to help you set that up.
Once you have stored your API key you will need to edit the ‘misp_config.ini’ file with the relevant information of where to obtain your key from:
- secret_name: <secret_name>
- region_name: <aws_region>
- API_key: <specified_key_name>
If no AWS data is provided and you have set the MISP flag in the arguments, the ioc_hunter.py file will prompt you for the relevant data when run.
Step 4: Set query variable data
If you are searching for a MISP eventid or tag, you do not need to specify the −−variable parameter.
In order to search for IOCs in the Data Lake we only need to specify the IOC_JSON variable. It will automatically search the entire timeframe available, and all endpoints contained. If you do not wish to use MISP to build the IOC JSON then you can manually create a JSON file of IOCs and pass that as raw json in the variable argument. It will need to adhere to the below schema:
Valid ‘indicator_type’ entries are:
- ip
- url
- domain
- port
- sha256
- filename, file_path, file_path_name, pathname
The data is a string value and can contain wildcards ‘%’.
Step 5: Hunt
Now we have done the groundwork we can now start hunting! Here is a simple example of using IOC Hunter for XDR to search for the IOCs for a specific MISP event id:
--api xdr --search_type saved --search_input "<saved_search_name>" --misp true --misp_type eventid --misp_val <eventid_value> --log_level INFO
As you can see this is like the live discover IOC Hunter command, the new difference being the parameter ‘−−api’ that differentiates it between the two query API endpoints. Using this new parameter along with the original parameters of filters and variables, which are covered in the previous blog post, you can continue to do Live Discover IOC Hunts.
More examples can be found in the GitHub docs here.
Output
Changes have been made to the output since the original IOC Hunter for Live Discover. Files are more clearly marked to differentiate between the two API endpoints and will be saved to the following location: <path_to_folder>\sophos_central_api_connector\output\query_results
Both XDR and Live Discover will produce the files:
- <api>_search_data_<timestamp>.json
- This contains details on the search run and on which tenants. It contains information on the overall outcome of the search.
- <api>_result_data_<timestamp>.json
- This contains any hits on the IOCs or provides details on the outcome of the saved search run. It will also clarify where no results were found on a tenant
Live Discover will also output:
- <api>_endpoint_data_<timestamp>.json
- This contains details on the endpoints that the query was run on, this will allow you to check which endpoints failed to run the query so they can be re-queried at a later point.
We hope this is a useful method to hunt for IOCs in your estate!
Happy hunting!
