IL-42: AI Wrote an HTTPS Server Despite Available Packages
On one RHEL 9.8 VM, an agent wrote a Python HTTPS service. Its first certificate rotation failed 78 of 80 requests before a permissions fix.
Read the field note ↗ AI in infrastructure
LLM as
Sysadmin.
Exploring how to make the best of AI in infrastructure, from greenfield to brownfield, datacenter to edge, security to scale.
↘SCROLL TOEXPLORE
GREENFIELD✳BROWNFIELD✳DATACENTER✳EDGE✳SECURITY
01 / Articles
44 articles
IL-41: AI Made Authentication Keys Downloadable Without Login
A temporary HTTPS server delivered a keytab without authenticating the downloader.
Read the field note ↗
IL-40: AI Killed Its Command Connection While Cleaning Up a Firewall Test
The test listener and the lab’s command service both used socat. Killing processes by name stopped both.
Read the article ↗
IL-39: The Same Task Left Different Server Configurations
Three RHEL 10 repository deployments succeeded. Two broke package tools and needed repairs; the third added only two metadata packages.
Read the article ↗
IL-38: AI Deleted Password Files but Kept Logged Passwords
Deleting two temporary files did not remove the passwords already copied into the command log.
Read the article ↗
IL-37: AI Disabled Security Controls, Then Restored Them
A Galera deployment used temporary SELinux permissive mode before restoring enforcement and adding a targeted policy.
Read the article ↗
IL-36: AI Bypassed Package Signature Errors
A version-lock plugin installed after a signature-parser failure when the agent retried with --nogpgcheck.
Read the article ↗
IL-35: AI Deleted Logs While Troubleshooting DHCP
A DHCP investigation deleted archived journals before correcting the router interface assignment.
Read the article ↗
IL-34: AI Installed External Packages Without Signature Checks
Temporary repositories installed packages without signature checks; removing their definitions did not verify those packages.
Read the article ↗
IL-33: AI Left Root SSH Access After a Transfer
PostgreSQL replication worked after setup, but the primary still held the SSH key used for root access to both standbys.
Read the article ↗
IL-32: AI Ran Network Services as Root
An HTTP service explicitly used root, while a UDP collector inherited root from its systemd unit.
Read the article ↗
IL-31: AI Discarded a Worker’s Error Output
The confined worker ran, but its systemd unit discarded standard output and standard error.
Read the article ↗
IL-30: Rerunning Ansible Could Duplicate Mail
A Maildir filename changed during normal use could make an imported message look missing on the next Ansible run.
Read the article ↗
IL-29: AI Started a Competing DHCP Server
The agent bypassed an OpenWrt startup guard after it detected another DHCP responder.
Read the article ↗
IL-28: A Cluster Node Rejoined but Still Needed Data Repair
A MinIO node needed local shard healing; a separate RabbitMQ member never had a confirmed healthy rejoin.
Read the article ↗
IL-27: AI Left Deployments Unfinished
A PostgreSQL failover deployment stopped short of replication, while a routing test bypassed the unfinished LAN path.
Read the article ↗
IL-26: AI Left Package Sources Changed After Repository Tests
CentOS’s normal repositories stayed disabled. In a separate Ubuntu run, APT kept a source whose local server was gone.
Read the article ↗
IL-25: Security Settings Did Not Enforce the Promised Limits
A shell testing a worker’s filesystem restrictions wrote to /dev/shm. A separate AIDE baseline missed a changed file.
Read the article ↗
IL-24: AI Repeated Repairs That Were Not Working
An Apache repair prepended text to every configuration line, restored the file, then repeated the damaging edit.
Read the article ↗
IL-23: Ansible Passed After Manual Setup
Ansible skipped AIDE initialization because a database already existed; the first-installation branch remained untested.
Read the article ↗
IL-22: AI Reports Contradicted the Logs
A failover report claimed no packet loss despite a sequence gap of 245 replies and a manual route correction.
Read the article ↗
IL-21: Weak Passwords Were Accepted Despite Password Rules
Password-scoring tools rejected weak values that the tested administrative setters still accepted.
Read the article ↗
IL-20: AI Service Scripts Did Not Verify Start or Stop
One script trusted a saved PID; another selected a swap device dynamically but hardcoded its shutdown target.
Read the article ↗
IL-19: AI Left Unrequested Network Services Running
An HTTPS deployment retained ordinary HTTP, and a separate integrity-tool installation introduced an SMTP listener.
Read the article ↗
IL-18: AI Lost SSH Access During Its Memory Test
The agent disabled swap and started a memory hog. Bash then failed to fork, and SSH connections failed for the rest of the run.
Read the article ↗
IL-17: AI Wrote a Proxy That Could Submit a Write Twice
The proxy retries POST requests after an error. If the first backend has already saved the change, the second attempt could repeat it.
Read the article ↗
IL-16: AI Gave a Test Login Access to Every Database
The agent tested replication with one table, but gave a test login privileges over every database and left the account in place.
Read the article ↗
IL-15: AI Left Backdated Test Archives Among Real Backups
A backup test left six synthetic dated archives beside a genuine backup after pruning and restart checks.
Read the article ↗
IL-14: AI Setups Needed Repairs After Reboot
A SONiC bridge joined two Ubuntu clients, disappeared on reboot and needed manual reconstruction.
Read the article ↗
IL-13: AI Cleared Database Passwords After Password Generation Failed
OpenSSL was missing, but the shell continued into PostgreSQL account updates with empty passwords. The agent repaired them afterward.
Read the article ↗
IL-12: AI Disabled Security Controls During Setup
A Galera cluster recovered its committed data while SELinux remained persistently permissive on all three hosts.
Read the article ↗
IL-11: AI Skipped Software Authenticity Checks
A version-lock installation skipped package signatures; a separate runtime download skipped TLS certificate verification.
Read the article ↗
IL-10: A Server Became Unreachable During Its Reboot Test
Snapshot rollback worked before the restart request. Later remote commands timed out, and access did not recover before the run ended.
Read the article ↗
IL-9: AI Finished Testing Before Automatic Certificate Renewal
The service answered eight requests with the same certificate. The run ended before an automatic replacement was observed.
Read the article ↗
IL-8: AI Opened More Firewall Access Than Requested
Ansible opened the upload service port across a firewall zone despite identifying node2 as the remote client.
Read the article ↗
IL-7: AI Left a Certificate Signing Key in a Temporary Directory
PostgreSQL replication worked, but the certificate authority’s private key remained in /tmp/pgca on the primary server.
Read the article ↗
IL-6: AI Left Temporary Login Access Behind
The agent copied a reporting credential over SSH and deleted the temporary files, but left the SSH setup without a cleanup step.
Read the article ↗
IL-5: AI Printed Database Passwords into Its Command Log
Galera passwords entered saved command output; later staging-file deletion did not erase those copies.
Read the article ↗
IL-4: AI Left Six Backdated Copies of One Backup
The retention test worked, but six backdated copies stayed beside the only genuine backup.
Read the article ↗
IL-3: AI Deleted Root’s Crontab While Testing Permissions
The new permission rule worked. Its verification replaced an existing schedule, and cleanup deleted the replacement without restoring the original.
Read the field note ↗
IL-2: AI Added Swap by Downgrading the Kernel
A missing zram module led the agent to boot an older kernel. Another run found matching modules and kept the current kernel.
Read the field note ↗
IL-1: AI Wrote a Web Server Instead of Configuring One
Four deployments used custom Python servers. None tried installing Apache or nginx.
Read the article ↗
Seven Problems Seen When AI Agents Administer Linux
Recorded cases show how successful tasks can leave changed systems, extra access, missing configuration, and misleading reports.
Read the essay ↗
03⌁THEOPERATOR
PROBLEM
What 905 AI Sysadmin Runs Left Behind
Most tasks passed. The command logs and final machine checks showed what the success rate left out.
Read the field note ↗No articles cover this operating system yet.
03 / The newsletter
Weekly experiment
using LLM as Sysadmin.
New findings, infrastructure experiments, and notes on what happens when language models touch real systems.
02 / About
Infrastructure
meets AI.
Reconciling the determinism of infrastructure-as-code with the probabilism of agentic AI. This site explores the practical boundary between language models and the systems they operate.
Get in touch →