5 open source tools for container security

As containers become an almost ubiquitous method of packaging and deploying applications, the instances of malware have increased. Securing containers is now a top priority for DevOps engineers. Fortunately, a number of open source programs are available that scan containers and container images. Let’s look at five such tools.

Anchore | Clair | Dagda | OpenSCAP | Sysdig Falco

Anchore

The Anchore engine is an open source project that inspects, analyzes, and certifies Docker images. Anchore is available as a Docker image that can be run standalone or with orchestration platforms such as Kubernetes. Anchore fetches security data from Anchore’s hosted cloud service. Anchore is also available as a Jenkins plugin, allowing you to integrate container image scanning as part of the CI/CD workflow.

Anchore Engine has a straightforward install process thanks to the Docker compose file. Installation instructions are available on the GitHub page as well as on the support Knowledge Base. The Anchore Engine provides the back-end/server-side component while for scanning the images, Anchore requires another component. The scanner can be in the form of a CLI tool such as a Anchore CLI or a Jenkins plugin, both of which are developed and maintained by Anchore.

To initiate a scan, add the image to the Anchore Engine with the following command:

anchore-cli image add python:3

This command submits the image to be scanned. Once submitted, Anchore will initiate a scan of the image. Unfortunately, there seems to be no way to know when the scan is completed, except by typing (or writing a script):

anchore-cli image get python:3 

to monitor the status. Once an image is scanned, use the following command to list the reported CVEs in the image:

anchore-cli image vuln python:3 os

Although the CLI reports CVE vulnerabilities, the vuln command can report non-OS vulnerabilities. Running it did not report any data in this example:

anchore-cli image vuln python:3
os: available
non-os: available

anchore-cli image vuln python:3 non-os
<no output>

Anchore scan can also report the artifacts present in the image, including Python packages, Ruby gems, os packages, and all other files on the filesystem:

anchore-cli image content python:3 python

With this data, Anchore lets you evaluate the image against policies to check for security compliance. To do this, type:

anchore-cli evaluate check python:3
Image Digest: sha256:9d49a3fdfd4198061e10241ee2ff9d75fb270c80747973360c22cd553f1e228c
Full Tag: docker.io/python:3
Status: fail
Last Eval: 2018-07-11T13:20:58Z
Policy ID: 2c53a13c-1765-11e8-82ef-23527761d060

By default, Anchore will only provide a summary of the evaluation. To obtain a full evaluation, you can pass the --detail flag, which provides the full evaluation:

anchore-cli evaluate check python:3 --detail
Image Digest: sha256:9d49a3fdfd4198061e10241ee2ff9d75fb270c80747973360c22cd553f1e228c
Full Tag: docker.io/python:3
Image ID: 17453243214e5dd215bdac0f00c639d82941e1d577ee959c842de73d533da572
Status: fail
Last Eval: 2018-07-11T13:22:12Z
Policy ID: 2c53a13c-1765-11e8-82ef-23527761d060
Final Action: stop
Final Action Reason: policy_evaluation

The gates, thresholds, and evaluations are performed against a policy whose ID is mentioned above. Examine the policy by typing:

anchore-cli policy get 2c53a13c-1765-11e8-82ef-23527761d060 --detail

This outputs a JSON of the policy. The open source Anchore Engine lets you define and activate custom policies, but it doesn’t come with a visual policy editor, so trying to edit the policies by hand can be tricky. Anchore also works well with private registries; add them using:

anchore-cli registry add <registry> <username> <password>

Anchore can also be configured to scan repositories and add any tags found in the repository. Once added, the Anchore engine will poll the registry periodically and schedule them to be analyzed.

Clair

An open source vulnerability scanner and static analysis tool for container images by CoreOS, Clair is the same tool that powers CoreOS’s container registry, Quay.io.

Clair regularly ingests vulnerability information from various sources and saves it in the database. Clair exposes APIs for clients to invoke and perform scans. That said, Clair is a “backend-only” tool and doesn’t come with scanners or a frontend for initiating a scan. There are a few third-party tools that integrate with Clair, but to scan images from a terminal as part of a deploy script, the only reasonable option seems to be klar by optiopay.

Clair’s install instructions can found on its Github repo and can be run as a container with Docker. Clair also comes with a Docker Compose file and a Helm Chart to simplify installation, or it can be compiled from source. These steps only bootstrap the server—to run a scan, you need a compatible frontend.

While Clair’s integrations page lists the options available, for this article, I looked at the available command-line utilities available. Of these, klar was the simplest and quickest. klar can be downloaded from its GitHub releases page or compiled from scratch.

Running klar is straightforward. To start a scan, the syntax is:

CLAIR_ADDR=<Clair server URL>  klar <image name to be scanned>

For instance, assuming that we used the Docker Compose file to bring up the Clair server and we want to scan python:3 Docker image, the command would be like this:

CLAIR_ADDR=localhost klar python:3

Klar will pull the Docker image (if not present on the host) and run the scan against Clair. When it's done, it will present a report like the following:

Analysing 9 layers

Got results from Clair API v3
Found 488 vulnerabilities
Unknown: 22
Negligible: 181
Low: 150
Medium: 113
High: 22

CVE-2017-12424: [High]

Found in: shadow [1:4.4-4.1]

Fixed By:
In shadow before 4.5, the newusers tool could be made to manipulate internal data structures in ways unintended by the authors. Malformed input may lead to crashes (with a buffer overflow or other memory corruption) or other unspecified behaviors. This crosses a privilege boundary in, for example, certain web-hosting environments in which a Control Panel allows an unprivileged user account to create subaccounts.
https://security-tracker.debian.org/tracker/CVE-2017-12424

Klar lets you customize its features via environment variables. Some noteworthy ones include:

CLAIR_OUTPUT: Lets you define the severity of the vulnerabilities klar displays. For instance, setting CLAIR_OUTPUT to Medium means klar will display only vulnerabilities that are tagged as medium severity or higher.

JSON_OUTPUT: klar will output the vulnerability report in JSON if this is set to true.

WHITELIST_FILE: klar will not report the CVEs listed here as a vulnerability. The is a YAML file; a sample configuration is available in klar’s repository.

DOCKER_USER / DOCKER_PASSWORD / DOCKER_TOKEN: If you have a registry that needs authentication, passing these lets klar pull the image using these credentials.

Dagda

Dagda performs static analysis of known vulnerabilities. It uses the ClamAV anti-virus engine to scan and detect trojans, viruses, and malware contained within Docker images. Dagda also integrates with Sysdig Falco to monitor running Docker containers for anomalies.

Dagda comes with a Docker Compose file as well, making it easy to evaluate. The Docker Compose file and related installation instructions are available in Dagda’s Github repository. Prior to running the Dagda CLI, you must set the Dagda server’s hostname and port. To do this, use the following commands:

export DAGDA_HOST='127.0.0.1'
export DAGDA_PORT=5000

Before using Dagda for analysis, you need to initialize the vulnerability database. To do this, use the following command:

python3 dagda.py vuln --init

This takes a bit of time and can be monitored by watching the result of the following command using the --init_status option:

watch python3 dagda.py vuln --init_status

Once the vulnerability database is ready, the status is updated:

1. Every 2.0s: python dagda.py vuln --init...  sabhat: Sun Jul 15 16:39:27 2018
2. 
3. {
4.     "status": "Updated",
5.     "timestamp": "2018-07-15 11:04:51.234453"
6. }
7.  Now, we can start using Dagda. 
8. Dagda lets us search for specific CVEs
9. 
10. python dagda.py vuln --cve_info CVE-2009-2890
11. [
12.     {
13.         "cveid": "CVE-2009-2890",
14.         "cvss_access_complexity": "Medium",
15.         "cvss_access_vector": "Network",
16.         "cvss_authentication": "None required",
17.         "cvss_availability_impact": "None",
18.         "cvss_base": 4.3,
19.         "cvss_confidentiality_impact": "None",
20.         "cvss_exploit": 8.6,
21.         "cvss_impact": 2.9,
22.         "cvss_integrity_impact": "Partial",
23.         "cvss_vector": [
24.             "AV:N",
25.             "AC:M",
26.             "Au:N",
27.             "C:N",
28.             "I:P",
29.             "A:N"
30.         ],
31.         "cweid": "CWE-79",
32.         "mod_date": "16-08-2017",
33.         "pub_date": "20-08-2009",
34.         "summary": "Cross-site scripting (XSS) vulnerability in results.php in PHP Scripts Now Riddles allows remote attackers to inject arbitrary web script or HTML via the searchquery parameter."
35.     }
36. ]

Dagda also lets you search for CVEs by product:

1. python dagda.py vuln --product openssh
2. [
3.     {
4.         "CVE-1999-1010": {
5.             "cveid": "CVE-1999-1010",
6.             "cvss_access_complexity": "Low",
7.             "cvss_access_vector": "Local access",
8.             "cvss_authentication": "None required",
9.             "cvss_availability_impact": "None",
10.             "cvss_base": 2.1,
11.             "cvss_confidentiality_impact": "Partial",
12.             "cvss_exploit": 3.9,
13.             "cvss_impact": 2.9,
14.             "cvss_integrity_impact": "None",
15.             "cvss_vector": [
16.                 "AV:L",
17.                 "AC:L",
18.                 "Au:N",
19.                 "C:P",
20.                 "I:N",
21.                 "A:N"
22.             ],
23.             "cweid": "CWE-0",
24.             "mod_date": "17-10-2016",
25.             "pub_date": "14-12-1999",
26.             "summary": "An SSH 1.2.27 server allows a client to use the \"none\" cipher, even if it is not allowed by the server policy."
27.         }

Scan a Docker image by using the following syntax:

1. python3 dagda.py check --docker_image python:3
2. {
3.     "id": "5b4b2ef44785ff0001b76e9d",
4.     "msg": "Accepted the analysis of <python:3>"
5. }

The ID obtained above can be used to fetch a detailed report using the following:

1. python dagda.py history python:3 --id 5b4b32ec4785ff000106c693
2. [
3.     {
4.         "id": "5b4b2ef44785ff0001b76e9d",
5.         "image_name": "python:3",
6.         "status": "Analyzing",
7.         "timestamp": "2018-07-15 11:24:36.866439"
8.     }
9. ]

The analysis can take some time, so be patient. Once analyzed, you can get a detailed report by using the following command:

1. python dagda.py history python:3 --id 5b4b32ec4785ff000106c693
2. [
3.     {
4.         "id": "5b4b32ec4785ff000106c693",
5.         "image_name": "python:3",
6.         "static_analysis": {
7.             "malware_binaries": [],
8.             "os_packages": {
9.                 "ok_os_packages": 388,
10.                 "os_packages_details": [
11.                     {
12.                         "is_false_positive": false,
13.                         "is_vulnerable": true,
14.                         "product": "bash",
15.                         "version": "4.4",
16.                         "vulnerabilities": [
17.                             {
18.                                 "CVE-2017-5932": {
19.                                     "cveid": "CVE-2017-5932",
20.                                     "cvss_access_complexity": "Low",
21.                                     "cvss_access_vector": "Local access",
22.                                     "cvss_authentication": "None required",
23.                                     "cvss_availability_impact": "Partial",
24.                                     "cvss_base": 4.6,
25.                                     "cvss_confidentiality_impact": "Partial",
26.                                     "cvss_exploit": 3.9,
27.                                     "cvss_impact": 6.4,
28.                                     "cvss_integrity_impact": "Partial",
29.                                     "cvss_vector": [
30.                                         "AV:L",
31.                                         "AC:L",
32.                                         "Au:N",
33.                                         "C:P",
34.                                         "I:P",
35.                                         "A:P"
36.                                     ],
37.                                     "cweid": "CWE-20",
38.                                     "mod_date": "31-03-2017",
39.                                     "pub_date": "27-03-2017",
40.                                     "summary": "The path autocompletion feature in Bash 4.4 allows local users to gain privileges via a crafted filename starting with a \" (double quote) character and a command substitution metacharacter."
41.                                 }
42.                             },
43.                             [....]
44.                         ]
45.                     },
46.                 [...]
47.                 ],
48.                 "total_os_packages": 416,
49.                 "vuln_os_packages": 28
50.             },
51.             "prog_lang_dependencies": {
52.                 "dependencies_details": {
53.                     "java": [],
54.                     "js": [
55.                         {
56.                             "is_false_positive": false,
57.                             "is_vulnerable": true,
58.                             "product": "jquery",
59.                             "product_file_path": "/usr/share/doc/libfreetype6/js/jquery-1.11.0.min.js",
60.                             "version": "1.11.0.min",
61.                             "vulnerabilities": []
62.                         }
63.                     ],
64.                     "nodejs": [],
65.                     "php": [],
66.                     "python": [],
67.                     "ruby": []
68.                 },
69.                 "vuln_dependencies": 1
70.             }
71.         },
72.         "status": "Completed",
73.         "timestamp": "2018-07-15 11:48:41.214345"
74.     }
75. ]

Although Dagda does support monitoring of containers, this requires Sysdig Falco to be running. Dagda doesn’t support scanning of repositories or registries, making it more suitable for on-demand scans than scheduled or automatic registry scans.

OpenSCAP

Security Content Automation Protoco (SCAP) enables automated vulnerability management, measurement, and policy compliance evaluation using a specific set of standards. OpenSCAP is an example of SCAP implementation. OpenSCAP provides a set of tools for scanning and compliance management, including oscap-docker, which can scan a container image.

oscap-docker lets you scan an image for xccdf (Extensible Configuration Checklist Description Format) compliance using the following command:

oscap-docker image <image-name> xccdf eval --report results.html --profile standard <path to xccdf definition file>

For example, to scan a CentOS image:

oscap-docker image centos xccdf eval --report results.html --profile standard /usr/share/xml/scap/ssg/content/ssg-centos7-xccdf.xml

The resulting output will look like this:

1. Title   Disable At Service (atd)
2. Rule    service_atd_disabled
3. Result  pass
4. 
5. Title   Disable Network Router Discovery Daemon (rdisc)
6. Rule    service_rdisc_disabled
7. Result  fail
8. 
9. Title   Disable Odd Job Daemon (oddjobd)
10. Rule    service_oddjobd_disabled
11. Result  pass
12. 
13. Title   Disable Apache Qpid (qpidd)
14. Rule    service_qpidd_disabled
15. Result  pass
16. 
17. Title   Disable Automatic Bug Reporting Tool (abrtd)
18. Rule    service_abrtd_disabled
19. Result  pass
20. 
21. Title   Disable ntpdate Service (ntpdate)
22. Rule    service_ntpdate_disabled
23. Result  notapplicable
24. 
25. Title   Verify and Correct File Permissions with RPM
26. Rule    rpm_verify_permissions
27. Result  fail
28. 
29. Title   Verify File Hashes with RPM
30. Rule    rpm_verify_hashes
31. Result  pass
32. 
33. Title   Ensure Software Patches Installed
34. Rule    security_patches_up_to_date
35. Result  notchecked

OpenSCAP’s CVE scan for container images seems to work only for RHEL images; for others, oscap-docker kept showing the message:

<image> is not based on RHEL

The scan wouldn’t proceed from that point.

Sysdig Falco

While it's not a pure container security or CVE scanning solution, Sysdig Falco deserves a mention. Sysdig Falco monitors our running Docker containers and provides insights into the behavior of containers and applications within containers. It provides a rich set of default notifications and alerts and lets you customize the rules for them.

Sysdig Falco works within containers as well as on the Linux host, but Sysdig recommends installing it on the host operating system. Instructions for installing Sysdig Falco on containers and on the host operating system are available below:

• Installing on containers

• Installing on host systems

Once it's installed, start Sysdig Falco:

sudo falco

Falco will load the rules and provide the following message when ready:

1. Falco initialized with configuration file /etc/falco/falco.yaml
2. Loading rules from file /etc/falco/falco_rules.yaml:
3. Loading rules from file /etc/falco/falco_rules.local.yaml:

Falco will display messages at different severity levels according to the rules:

1. Debug Shell spawned by untrusted binary (user=root shell=sh parent=httpd cmdline=sh -c ls > /dev/null pcmdline=httpd --action spawn_shell --interval 0 --once gparent=event_generator ggparent=docker-containe gggparent=docker-containe ggggparent=dockerd)
2. Notice Known system binary sent/received network traffic (user=root command=sha1sum --action network_activity --interval 0 --once connection=172.17.0.2:56852->10.2.3.4:8192)
3. Informational System user ran an interactive command (user=daemon command=login )
4. Error File below a known binary directory opened for writing (user=root command=event_generator  file=/bin/created-by-event-generator-sh parent=docker-containe pcmdline=docker-containe -namespace moby -workdir /var/lib/docker/containerd/daemon/io.containerd.runtime.v1.linux/moby/82dc57b2e8b551e09a07a694763be59193f9b29e64773040a1710560d5570927 -address /var/run/docker/containerd/docker-containerd.sock -containerd-binary /usr/bin/docker-containerd -runtime-root /var/run/docker/runtime-runc gparent=docker-containe)
5. Error File below /etc opened for writing (user=root command=event_generator  parent=docker-containe pcmdline=docker-containe -namespace moby -workdir /var/lib/docker/containerd/daemon/io.containerd.runtime.v1.linux/moby/82dc57b2e8b551e09a07a694763be59193f9b29e64773040a1710560d5570927 -address /var/run/docker/containerd/docker-containerd.sock -containerd-binary /usr/bin/docker-containerd -runtime-root /var/run/docker/runtime-runc file=/etc/created-by-event-generator-sh program=event_generator gparent=docker-containe ggparent=dockerd gggparent=systemd)

What good are these alerts? First, they can be forwarded to a SIEM tool like Splunk and we can further create/emit alerts based on the count and frequency of these alerts. Falco can also call a program and send the alerts to that program. For example, we can configure Falco to send notifications to Slack by making a cURL call to the webhook endpoint.

Conclusion

While most open source scanners are small components of the larger container security platform, they perform admirably well. Anchore and Clair provide a complete solution for scanning registries and repositories as new images are pushed. Dagda can be a bit slow in scanning, but it does the job. And for RHEL-based images, OpenSCAP might integrate well.