Amazon Linux 2 score is always 50, with or without hardening in place

[ceso]

Describe the bug
I'm building an Amazon Linux 2, after the scan kicks in and some policies are flagged as Fail/Pass, the scan is always scored as 50. This happens regardless of whether hardening policies are in place (they are being shown as Pass by the scan), or if no extra hardening is in place. No matter what's done, the score is always 50.
This has happened either by using the latest v11.7.3, v11.5.0, or v10.9.2 (I haven't tried with other versions). The version of Packer I am using is v1.11.0, and the version of the Amazon provider is amazon-ebs v1.3.2.

To Reproduce
Steps to reproduce the behavior:

If this code from the examples https://github.com/mondoohq/packer-plugin-cnspec/blob/main/examples/aws/amazon-linux-2.pkr.hcl is used, the parameter score_threshold added (set to 50 and then to 80 for example), and a build attempt is executed with both one with hardening in place and one without, the score should always be 50.

Expected behavior
Correct behavior should be, if the image has been hardened the score should reflect such a thing. An image with multiple Pass should score higher than one with multiple Fails.

Screenshots or CLI Output

Here's a CLI output generating an image without any hardening measure in place (49 Fail):

    amazon-ebs.goldenbase: activated sudo
    amazon-ebs.goldenbase: detected packer build via ssh
    amazon-ebs.goldenbase: no configuration provided
    amazon-ebs.goldenbase: successfully updated OS provider
    amazon-ebs.goldenbase: use OS provider version 11.2.8 (/home/ceso/.config/mondoo/providers/os)
    amazon-ebs.goldenbase: scan packer build in incognito mode
    amazon-ebs.goldenbase: Asset: i-0216c37ff10b2fad7
    amazon-ebs.goldenbase: --------------------------
    amazon-ebs.goldenbase: 
    amazon-ebs.goldenbase: Checks:
    amazon-ebs.goldenbase: ✕ Fail:   25  Ensure ICMP redirects are not accepted
    amazon-ebs.goldenbase: ✕ Fail:   40  Ensure suspicious packets are logged
    amazon-ebs.goldenbase: ✕ Fail:   50  Ensure events that modify the system's network environment are collected
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure address space layout randomization (ASLR) is enabled
    amazon-ebs.goldenbase: ✕ Fail:   50  Ensure the audit configuration is immutable
    amazon-ebs.goldenbase: ! Error:      Ensure SSH MaxAuthTries is set to 4 or less
    amazon-ebs.goldenbase: ✕ Fail:   50  Ensure system administrator actions (sudolog) are collected
    amazon-ebs.goldenbase: ! Error:      Ensure SSH LoginGraceTime is set to one minute or less
    amazon-ebs.goldenbase: ✕ Fail:   50  Ensure events that modify date and time information are collected
    amazon-ebs.goldenbase: ✕ Fail:   20  Ensure sudo logging is enabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure tftp server is stopped and not enabled
    amazon-ebs.goldenbase: ✕ Fail:       Ensure SSH PermitEmptyPasswords is disabled
    amazon-ebs.goldenbase: ✕ Fail:   50  Ensure discretionary access control permission modification events are collected
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure CUPS is stopped and not enabled
    amazon-ebs.goldenbase: ✕ Fail:       Ensure SSH LogLevel is appropriate
    amazon-ebs.goldenbase: ✕ Fail:    0  Ensure secure permissions on /etc/passwd- are set
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure audit log storage size is configured
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure shadow group is empty
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure IP forwarding is disabled
    amazon-ebs.goldenbase: ✕ Fail:   40  Ensure SSH access is limited
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure rsyslog is installed
    amazon-ebs.goldenbase: ✕ Fail:   50  Ensure successful file system mounts are collected
    amazon-ebs.goldenbase: ✕ Fail:   30  Ensure SSH PermitUserEnvironment is disabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure secure permissions on /etc/shadow are set
    amazon-ebs.goldenbase: ✕ Fail:   50  Ensure journald is configured to send logs to rsyslog
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure DNS server is stopped and not enabled
    amazon-ebs.goldenbase: ✕ Fail:   20  Ensure secure permissions on all log files are set
    amazon-ebs.goldenbase: ✕ Fail:   50  Ensure session initiation information is collected
    amazon-ebs.goldenbase: ✕ Fail:    0  Ensure secure permissions on SSH private host key files are set
    amazon-ebs.goldenbase: ✕ Fail:   50  Ensure filesystem integrity is regularly checked
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure DHCP server is stopped and not enabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure no duplicate user names exist
    amazon-ebs.goldenbase: ✕ Fail:   50  Ensure journald is configured to compress large log files
    amazon-ebs.goldenbase: ✕ Fail:   25  Ensure IPv6 router advertisements are not accepted
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure secure permissions on /etc/gshadow are set
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure each user is a member of a group
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure TCP SYN Cookies is enabled
    amazon-ebs.goldenbase: ✕ Fail:   50  Ensure changes to system administration scope (sudoers) is collected
    amazon-ebs.goldenbase: ✕ Fail:   50  Ensure unsuccessful unauthorized file access attempts are collected
    amazon-ebs.goldenbase: ! Error:      Ensure SSH Idle Timeout Interval is configured
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure bogus ICMP responses are ignored
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure Samba is stopped and not enabled
    amazon-ebs.goldenbase: ✕ Fail:   50  Ensure SSH X11 forwarding is disabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure FTP server is stopped and not enabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure vulnerable OpenSSL version 3.0.0 - 3.0.6 are not installed
    amazon-ebs.goldenbase: ✕ Fail:   20  Ensure only strong MAC algorithms are used
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure source routed packets are not accepted
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure NIS server is stopped and not enabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure broadcast ICMP requests are ignored
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure rsyslog Service is enabled
    amazon-ebs.goldenbase: ✕ Fail:   50  Ensure login and logout events are collected
    amazon-ebs.goldenbase: ✕ Fail:   30  Ensure SSH HostbasedAuthentication is disabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure rsync service is stopped and not enabled
    amazon-ebs.goldenbase: ✕ Fail:   25  Ensure secure ICMP redirects are not accepted
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure no duplicate GIDs exist
    amazon-ebs.goldenbase: ✕ Fail:    0  Ensure SSH root login is disabled or set to prohibit-password
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure secure permissions on /etc/passwd are set
    amazon-ebs.goldenbase: ✕ Fail:   50  Ensure events that modify the system's Mandatory Access Controls are collected
    amazon-ebs.goldenbase: ✕ Fail:   40  Ensure rsyslog default file permissions configured
    amazon-ebs.goldenbase: ✓ Pass:       Ensure disk usage is under 80%
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure Reverse Path Filtering is enabled
    amazon-ebs.goldenbase: ✕ Fail:   60  Ensure system is disabled when audit logs are full
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure HTTP servers are stopped and not enabled
    amazon-ebs.goldenbase: ✕ Fail:   50  Ensure journald is configured to write logfiles to persistent disk
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure LDAP server is stopped and not enabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure HTTP Proxy server is stopped and not enabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure rsh server is stopped and not enabled
    amazon-ebs.goldenbase: ✓ Pass:       Ensure secure permissions on SSH public host key files are set
    amazon-ebs.goldenbase: ✕ Fail:    0  Ensure only strong ciphers are used
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure SNMP server is stopped and not enabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure no duplicate group names exist
    amazon-ebs.goldenbase: ✕ Fail:    0  Ensure that strong Key Exchange algorithms are used
    amazon-ebs.goldenbase: ✕ Fail:   60  Ensure audit logs are not automatically deleted
    amazon-ebs.goldenbase: ✕ Fail:    0  Ensure EDR Agent is installed
    amazon-ebs.goldenbase: ✕ Fail:       Ensure SSH IgnoreRhosts is enabled
    amazon-ebs.goldenbase: ✕ Fail:   50  Ensure file deletion events by users are collected
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure system accounts are non-login
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure all GIDs in /etc/passwd exist in /etc/group
    amazon-ebs.goldenbase: ✕ Fail:   40  Ensure NFS and RPC are stopped and not enabled
    amazon-ebs.goldenbase: ✕ Fail:   50  Ensure auditing for processes that start prior to auditd is enabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure Avahi server is stopped and not enabled
    amazon-ebs.goldenbase: ✕ Fail:   20  Ensure access to the su command is restricted
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure mail transfer agent is configured for local-only mode
    amazon-ebs.goldenbase: ✕ Fail:    0  Ensure secure permissions on /etc/group- are set
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure X Window System is not installed
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure secure permissions on /etc/group are set
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure auditd service is enabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure talk server is stopped and not enabled
    amazon-ebs.goldenbase: ✕ Fail:   40  Ensure Advanced Intrusion Detection Environment (AIDE) is installed
    amazon-ebs.goldenbase: ✕ Fail:   70  Ensure SSH warning banner is configured
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure core dumps are restricted
    amazon-ebs.goldenbase: ✕ Fail:   50  Ensure kernel module loading and unloading is collected
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure IMAP and POP3 server is stopped and not enabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure secure permissions on /etc/shadow- are set
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure root group is empty
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure default group for the root account is GID 0
    amazon-ebs.goldenbase: ✕ Fail:   50  Ensure events that modify user/group information are collected
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure secure permissions on /etc/gshadow- are set
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure secure permissions on /etc/ssh/sshd_config are set
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure telnet server is stopped and not enabled
    amazon-ebs.goldenbase: ✕ Fail:   20  Ensure SSH Protocol is set to 2
    amazon-ebs.goldenbase: ✕ Fail:   25  Ensure packet redirect sending is disabled
    amazon-ebs.goldenbase: ✓ Pass:       Ensure memory usage is under 80%
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure UID_MIN is set to 1000
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure prelink is disabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure no duplicate UIDs exist
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure auditd is installed
    amazon-ebs.goldenbase: 
    amazon-ebs.goldenbase: 
    amazon-ebs.goldenbase: Scanned 1 asset
    amazon-ebs.goldenbase: 
    amazon-ebs.goldenbase: Amazon Linux 2
    amazon-ebs.goldenbase:     [50/100]    i-0216c37ff10b2fad7

and here, it's an output with some hardening in place (notice most of the scoring is a Pass, only 11 are a Fail) and despite this, the final score remains unchanged):

    amazon-ebs.goldenbase: activated sudo
    amazon-ebs.goldenbase: detected packer build via ssh
    amazon-ebs.goldenbase: no configuration provided
    amazon-ebs.goldenbase: successfully updated OS provider
    amazon-ebs.goldenbase: use OS provider version 11.2.8 (/home/ceso/.config/mondoo/providers/os)
    amazon-ebs.goldenbase: scan packer build in incognito mode
    amazon-ebs.goldenbase: Asset: i-06457369078b227fb
    amazon-ebs.goldenbase: --------------------------
    amazon-ebs.goldenbase: 
    amazon-ebs.goldenbase: Checks:
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure mail transfer agent is configured for local-only mode
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure talk server is stopped and not enabled
    amazon-ebs.goldenbase: ✕ Fail:   20  Ensure secure permissions on all log files are set
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure IPv6 router advertisements are not accepted
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure secure permissions on /etc/gshadow are set
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure Samba is stopped and not enabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure secure permissions on /etc/ssh/sshd_config are set
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure IMAP and POP3 server is stopped and not enabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure no duplicate GIDs exist
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure events that modify the system's Mandatory Access Controls are collected
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure LDAP server is stopped and not enabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure Reverse Path Filtering is enabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure secure permissions on /etc/group- are set
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure NIS server is stopped and not enabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure source routed packets are not accepted
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure secure permissions on /etc/shadow are set
    amazon-ebs.goldenbase: ✕ Fail:   40  Ensure Advanced Intrusion Detection Environment (AIDE) is installed
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure SNMP server is stopped and not enabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure events that modify date and time information are collected
    amazon-ebs.goldenbase: ✓ Pass:       Ensure memory usage is under 80%
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure no duplicate group names exist
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure Avahi server is stopped and not enabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure secure permissions on /etc/passwd are set
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure system administrator actions (sudolog) are collected
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure session initiation information is collected
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure CUPS is stopped and not enabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure file deletion events by users are collected
    amazon-ebs.goldenbase: ✕ Fail:    0  Ensure that strong Key Exchange algorithms are used
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure login and logout events are collected
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure SSH LoginGraceTime is set to one minute or less
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure SSH MaxAuthTries is set to 4 or less
    amazon-ebs.goldenbase: ✕ Fail:   60  Ensure audit logs are not automatically deleted
    amazon-ebs.goldenbase: ✕ Fail:   50  Ensure filesystem integrity is regularly checked
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure address space layout randomization (ASLR) is enabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure changes to system administration scope (sudoers) is collected
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure secure permissions on /etc/gshadow- are set
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure rsync service is stopped and not enabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure all GIDs in /etc/passwd exist in /etc/group
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure SSH PermitEmptyPasswords is disabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure audit log storage size is configured
    amazon-ebs.goldenbase: ✕ Fail:   40  Ensure SSH access is limited
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure each user is a member of a group
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure secure permissions on SSH private host key files are set
    amazon-ebs.goldenbase: ✕ Fail:   60  Ensure system is disabled when audit logs are full
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure rsyslog default file permissions configured
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure events that modify user/group information are collected
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure DNS server is stopped and not enabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure secure permissions on /etc/shadow- are set
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure only strong MAC algorithms are used
    amazon-ebs.goldenbase: ✓ Pass:       Ensure disk usage is under 80%
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure telnet server is stopped and not enabled
    amazon-ebs.goldenbase: ✓ Pass:       Ensure SSH PermitUserEnvironment is disabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure the audit configuration is immutable
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure core dumps are restricted
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure SSH Protocol is set to 2
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure SSH root login is disabled or set to prohibit-password
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure rsyslog is installed
    amazon-ebs.goldenbase: ✓ Pass:       Ensure SSH X11 forwarding is disabled
    amazon-ebs.goldenbase: ✕ Fail:   20  Ensure access to the su command is restricted
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure system accounts are non-login
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure shadow group is empty
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure auditd service is enabled
    amazon-ebs.goldenbase: ✕ Fail:   50  Ensure discretionary access control permission modification events are collected
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure ICMP redirects are not accepted
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure secure ICMP redirects are not accepted
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure successful file system mounts are collected
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure no duplicate UIDs exist
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure SSH LogLevel is appropriate
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure X Window System is not installed
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure broadcast ICMP requests are ignored
    amazon-ebs.goldenbase: ✓ Pass:       Ensure SSH IgnoreRhosts is enabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure secure permissions on /etc/passwd- are set
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure root group is empty
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure bogus ICMP responses are ignored
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure vulnerable OpenSSL version 3.0.0 - 3.0.6 are not installed
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure tftp server is stopped and not enabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure no duplicate user names exist
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure TCP SYN Cookies is enabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure journald is configured to compress large log files
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure default group for the root account is GID 0
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure HTTP Proxy server is stopped and not enabled
    amazon-ebs.goldenbase: ✓ Pass:       Ensure secure permissions on SSH public host key files are set
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure FTP server is stopped and not enabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure IP forwarding is disabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure HTTP servers are stopped and not enabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure suspicious packets are logged
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure DHCP server is stopped and not enabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure SSH warning banner is configured
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure events that modify the system's network environment are collected
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure journald is configured to write logfiles to persistent disk
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure rsyslog Service is enabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure journald is configured to send logs to rsyslog
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure auditd is installed
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure sudo logging is enabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure packet redirect sending is disabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure NFS and RPC are stopped and not enabled
    amazon-ebs.goldenbase: ✕ Fail:   50  Ensure auditing for processes that start prior to auditd is enabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure secure permissions on /etc/group are set
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure prelink is disabled
    amazon-ebs.goldenbase: ✓ Pass:       Ensure SSH HostbasedAuthentication is disabled
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure kernel module loading and unloading is collected
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure UID_MIN is set to 1000
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure rsh server is stopped and not enabled
    amazon-ebs.goldenbase: ✕ Fail:    0  Ensure EDR Agent is installed
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure only strong ciphers are used
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure unsuccessful unauthorized file access attempts are collected
    amazon-ebs.goldenbase: ✓ Pass:  100  Ensure SSH Idle Timeout Interval is configured
    amazon-ebs.goldenbase: 
    amazon-ebs.goldenbase: 
    amazon-ebs.goldenbase: Scanned 1 asset
    amazon-ebs.goldenbase: 
    amazon-ebs.goldenbase: Amazon Linux 2
    amazon-ebs.goldenbase:     [50/100]    i-06457369078b227fb
    amazon-ebs.goldenbase: 
    amazon-ebs.goldenbase:

Desktop (please complete the following information):

• OS: Ubuntu
• OS Version: 22.04
• Browser if applicable: N/A
• Browser Version: N/A

Additional context
The same behavior happens either by running Packer locally in my machine or by running from this Docker container: https://hub.docker.com/r/hashicorp/packer.

[chris-rock]

The output of [50/100] does not show the amount of checks passed but the score achieved.

[chris-rock]

While thinking about this, I think we want to improve the CLI output to show both the amount of passed checks and the achieved asset score.

[ceso]

Mmm but still though, why having non hardening or having hardening, achieve exactly the same score? I would expect that if some hardening is in place, the achieved score has improved, not keep the same

[chris-rock]

I am going to have a detailed look. Can you provide the hardening that you applied?

[ceso]

Every hardening measure applied, was taken from the remediations defined in the cnspec-core policies, these defined here: https://github.com/mondoohq/cnspec-policies/blob/main/core/mondoo-linux-security.mql.yaml

The remediations applied are the next ones, keep in mind that I am actually wrapping them inside a script, but these is what's being added/executed:

# === Ensure secure permissions on /etc/passwd- are set ===
chown root:root /etc/passwd-
chmod og-rwx /etc/passwd-
# === Ensure secure permissions on /etc/group- are set ===
chown root:root /etc/group-
chmod 600 /etc/group-
# === Ensure secure permissions on SSH private host key files are set ===
find /etc/ssh -xdev -type f -name 'ssh_host_*_key' -exec chown root:ssh_keys {} \;
find /etc/ssh -xdev -type f -name 'ssh_host_*_key' -exec chmod 0600 {} \;
# === Ensure secure permissions on SSH public host key files are set ===
find /etc/ssh -xdev -type f -name 'ssh_host_*_key.pub' -exec chmod 0644 {} \;
find /etc/ssh -xdev -type f -name 'ssh_host_*_key.pub' -exec chown root:root {} \;
# === Ensure NFS and RPC are stopped and not enabled ===
for i in stop mask; do for j in nfs rpcbind.service rpcbind.socket; do systemctl $i $j; done; done
# === Ensure journald is configured to send logs to rsyslog ===
sed -i '/^#ForwardToSyslog/ s/^#//' /etc/systemd/journald.conf
# === Ensure journald is configured to compress large log files ===
sed -i '/^#Compress/ s/^#//' /etc/systemd/journald.conf
# === Ensure journald is configured to write logfiles to persistent disk ===
sed -i 's/^#Storage=.*$/Storage=persistent/g' /etc/systemd/journald.conf
# === Ensure auditing for processes that start prior to auditd is enabled ===
echo 'GRUB_CMDLINE_LINUX="audit=1"' >> /etc/default/grub
grub2-mkconfig -o /boot/grub2/grub.cfg
# set proper permissions to auditd rules files
chown -R root:root /etc/audit/rules.d
# set proper permissions to sshd_config
chown root:root /etc/ssh/sshd_config
chmod 600 /etc/ssh/sshd_config
# === Ensure secure permissions on all log files are set ===
find /var/log/ -type f -exec chmod g-wx,o-rwx "{}" +

For auditd for example these are (these is the total, they are separated into files):

# === Ensure unsuccessful unauthorized file access attempts are collected ===
-a always,exit -F arch=b64 -S creat -S open -S openat -S truncate -S ftruncate -F exit=-EACCES -F auid>=1000 -F auid!=4294967295 -k access
-a always,exit -F arch=b32 -S creat -S open -S openat -S truncate -S ftruncate -F exit=-EACCES -F auid>=1000 -F auid!=4294967295 -k access
-a always,exit -F arch=b64 -S creat -S open -S openat -S truncate -S ftruncate -F exit=-EPERM -F auid>=1000 -F auid!=4294967295 -k access
-a always,exit -F arch=b32 -S creat -S open -S openat -S truncate -S ftruncate -F exit=-EPERM -F auid>=1000 -F auid!=4294967295 -k access
# === Ensure file deletion events by users are collected ===
-a always,exit -F arch=b64 -S unlink -S unlinkat -S rename -S renameat -F auid>=1000 -F auid!=4294967295 -k delete
-a always,exit -F arch=b32 -S unlink -S unlinkat -S rename -S renameat -F auid>=1000 -F auid!=4294967295 -k delete
# === Ensure events that modify user/group information are collected ===
-w /etc/group -p wa -k identity
-w /etc/passwd -p wa -k identity
-w /etc/gshadow -p wa -k identity
-w /etc/shadow -p wa -k identity
-w /etc/security/opasswd -p wa -k identity
# === Ensure login and logout events are collected ===
-w /var/log/lastlog -p wa -k logins
-w /var/log/tallylog -p wa -k logins
-w /var/run/faillock -p wa -k logins
# === Ensure events that modify the system's Mandatory Access Controls are collected ===
-w /etc/selinux/ -p wa -k MAC-policy
-w /usr/share/selinux/ -p wa -k MAC-policy
# === Ensure kernel module loading and unloading is collected ===
-w /sbin/insmod -p x -k modules
-w /sbin/rmmod -p x -k modules
-w /sbin/modprobe -p x -k modules
-a always,exit -F arch=b64 -S init_module -S delete_module -k modules
# === Ensure successful file system mounts are collected ===
-a always,exit -F arch=b64 -S mount -F auid>=1000 -F auid!=4294967295 -k mounts
-a always,exit -F arch=b32 -S mount -F auid>=1000 -F auid!=4294967295 -k mounts
# === Ensure discretionary access control permission modification events are collected ===
-a always,exit -F arch=b64 -S chmod -S fchmod -S fchmodat -F auid>=1000 -F auid!=4294967295 -k perm_mod
-a always,exit -F arch=b32 -S chmod -S fchmod -S fchmodat -F auid>=1000 -F auid!=4294967295 -k perm_mod
-a always,exit -F arch=b64 -S chown -S fchown -S fchownat -S lchown -F auid>=1000 -F auid!=4294967295 -k perm_mod
-a always,exit -F arch=b32 -S chown -S fchown -S fchownat -S lchown -F auid>=1000 -F auid!=4294967295 -k perm_mod
-a always,exit -F arch=b64 -S setxattr -S lsetxattr -S fsetxattr -S removexattr -S lremovexattr -S fremovexattr -F auid>=1000 -F auid!=4294967295 -k perm_mod
-a always,exit -F arch=b32 -S setxattr -S lsetxattr -S fsetxattr -S removexattr -S lremovexattr -S fremovexattr -F auid>=1000 -F auid!=4294967295 -k perm_mod
# === Ensure changes to system administration scope (sudoers) is collected ===
-w /etc/sudoers -p wa -k scope
-w /etc/sudoers.d -p wa -k scope
# === Ensure session initiation information is collected ===
-w /var/run/utmp -p wa -k session
-w /var/log/wtmp -p wa -k logins
-w /var/log/btmp -p wa -k logins
# === Ensure events that modify the system's network environment are collected ===
-a always,exit -F arch=b64 -S sethostname -S setdomainname -k system-locale
-a always,exit -F arch=b32 -S sethostname -S setdomainname -k system-locale
-w /etc/issue -p wa -k system-locale
-w /etc/issue.net -p wa -k system-locale
-w /etc/hosts -p wa -k system-locale
-w /etc/sysconfig/network -p wa -k system-locale
# === Ensure events that modify date and time information are collected ===
-a always,exit -F arch=b64 -S adjtimex -S settimeofday -k time-change
-a always,exit -F arch=b32 -S adjtimex -S settimeofday -S stime -k time-change
-a always,exit -F arch=b64 -S clock_settime -k time-change
-a always,exit -F arch=b32 -S clock_settime -k time-change
-w /etc/localtime -p wa -k time-change
# === Ensure the audit configuration is immutable ===
-e 2
# === Ensure system administrator actions (sudolog) are collected ===
-w /var/log/sudo.log -p wa -k actions

For sysctl:

# === Ensure packet redirect sending is disabled ===
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
# === Ensure secure ICMP redirects are not accepted ===
net.ipv4.conf.all.secure_redirects = 0
net.ipv4.conf.default.secure_redirects = 0
# === Ensure ICMP redirects are not accepted ###
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv6.conf.all.accept_redirects = 0
net.ipv6.conf.default.accept_redirects = 0
# === Ensure suspicious packets are logged ===
net.ipv4.conf.all.log_martians = 1
net.ipv4.conf.default.log_martians = 1
# ===  Ensure IPv6 router advertisements are not accepted ===
net.ipv6.conf.all.accept_ra = 0
net.ipv6.conf.default.accept_ra = 0

For rsyslog (dropped under /etc/rsyslog.d):

$FileCreateMode 0640
$umask 0077

For sudo (dropped under /etc/sudoers.d):

# === Ensure sudo logging is enabled ===
Defaults log_host, log_year, logfile="/var/log/sudo.log"

And /etc/ssh/sshd_config

# Package generated configuration file
# See the sshd_config(5) manpage for details

# What ports, IPs and protocols we listen for
Port 22
# Use these options to restrict which interfaces/protocols sshd will bind to
#ListenAddress ::
#ListenAddress 0.0.0.0
Protocol 2
# HostKeys for protocol version 2
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
HostKey /etc/ssh/ssh_host_ed25519_key
#Privilege Separation is turned on for security
UsePrivilegeSeparation yes

# Logging
SyslogFacility AUTH
LogLevel INFO

# Authentication:
LoginGraceTime 60
PermitRootLogin no
StrictModes yes

ClientAliveInterval 300
ClientAliveCountMax 0

PubkeyAuthentication yes
AuthorizedKeysFile	%h/.ssh/authorized_keys

# Don't read the user's ~/.rhosts and ~/.shosts files
IgnoreRhosts yes
# similar for protocol version 2
HostbasedAuthentication no
# Uncomment if you don't trust ~/.ssh/known_hosts for RhostsRSAAuthentication
#IgnoreUserKnownHosts yes

# To enable empty passwords, change to yes (NOT RECOMMENDED)
PermitEmptyPasswords no

# Change to yes to enable challenge-response passwords (beware issues with
# some PAM modules and threads)
ChallengeResponseAuthentication no

# Change to no to disable tunnelled clear text passwords
PasswordAuthentication no

# Specifies whether ~/.ssh/environment and environment options in ~/.ssh/authorized_keys are processed by sshd
PermitUserEnvironment no

# Kerberos options
#KerberosAuthentication no
#KerberosGetAFSToken no
#KerberosOrLocalPasswd yes
#KerberosTicketCleanup yes

# GSSAPI options
#GSSAPIAuthentication no
#GSSAPICleanupCredentials yes

X11Forwarding no
X11DisplayOffset 10
PrintMotd no
PrintLastLog yes
TCPKeepAlive yes
#UseLogin no

#MaxStartups 10:30:60
Banner /etc/issue.net

# Allow client to pass locale environment variables
AcceptEnv LANG LC_*

Subsystem sftp /usr/lib/openssh/sftp-server

# Set this to 'yes' to enable PAM authentication, account processing,
# and session processing. If this is enabled, PAM authentication will
# be allowed through the ChallengeResponseAuthentication and
# PasswordAuthentication.  Depending on your PAM configuration,
# PAM authentication via ChallengeResponseAuthentication may bypass
# the setting of "PermitRootLogin without-password".
# If you just want the PAM account and session checks to run without
# PAM authentication, then enable this but set PasswordAuthentication
# and ChallengeResponseAuthentication to 'no'.
UsePAM yes

# Ensure that strong Key Exchange algorithms are used
KexAlgorithms curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp521,ecdh-sha2-nistp384,ecdh-sha2-nistp256,diffie-hellman-group-exchange-sha256

# Ensure only approved MAC algorithms are used
MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512,hmac-sha2-256,umac-128@openssh.com

# Ensure only approved ciphers are used
Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr,aes192-ctr,aes128-ctr

# Limit the number of authentication attempts to avoid Brute force attacks
MaxAuthTries 4