Security Logs Should Explain Events Without Exposing Secrets

Detailed practical guide

Prepare your workspace and inputs

Save as security_log.py and run python3 security_log.py. The fake token DEMO_TOKEN_7391 exists only in the input fixture. The request ID includes a newline to test record framing.

Platform context: Python 3.10+; standard-library JSON and in-memory output. No real secrets are used.

Scenario to reproduce or investigate

An integration error log needs an event time, request ID, identity reference and outcome, but dumping the whole request can expose credentials and break one-record-per-line ingestion.

1. Specify the investigation fields

   Allow event_time_utc, event_type, request_id, principal_id and outcome. Keep the entire request body, authorization header and exception object out of this event path. Use a stable event type and UTC-aware timestamp.

   Command, configuration or input

   Allowed fields: event_time_utc, event_type, request_id, principal_id, outcome
   Excluded input fields: authorization, password, body

   Expected result · ผลที่ควรเห็น

   An analyst can correlate the event without needing a plaintext credential or full business payload.

   If it fails · เมื่อไม่ผ่าน

   If an integration needs more context, add a narrowly defined, reviewed field instead of allowing arbitrary dictionaries into the log.

2. Serialize one bounded record and test the excluded fields

   The full program validates field types, caps text lengths and uses JSON escaping. The fake token is never passed into the output record. JSON encoding makes the malicious newline part of a string rather than a second event.

   Save and run this example

   import json
   from datetime import datetime, timezone
   
   ALLOWED = {"event_time_utc", "event_type", "request_id", "principal_id", "outcome"}
   
   def event(payload):
       record = {"event_time_utc": datetime.now(timezone.utc).isoformat()}
       for field in ALLOWED - {"event_time_utc"}:
           value = payload.get(field)
           if not isinstance(value, str) or not value or len(value) > 100:
               raise ValueError("invalid event field: " + field)
           record[field] = value
       return json.dumps(record, ensure_ascii=True, sort_keys=True)
   
   raw = {"event_type": "access_denied", "request_id": "r1\nFAKE_EVENT",
          "principal_id": "user-17", "outcome": "denied",
          "authorization": "Bearer DEMO_TOKEN_7391", "password": "demo"}
   line = event(raw)
   decoded = json.loads(line)
   assert set(decoded) == ALLOWED
   assert "DEMO_TOKEN_7391" not in line and "authorization" not in line
   assert len(line.splitlines()) == 1
   assert decoded["request_id"] == "r1\nFAKE_EVENT"
   print(line)
   print("PASS: 5 approved fields; fake secret excluded; one physical record")
   

   Expected result · ผลที่ควรเห็น

   One JSON event is printed, followed by PASS. The secret is absent and the encoded request ID does not create a second physical log line.

   If it fails · เมื่อไม่ผ่าน

   If the token appears, inspect any logging middleware or exception handler that also logs the raw request. This event function cannot sanitize an independent raw-payload log elsewhere.

3. Verify the actual collector and access boundary

   Send the same fake-token fixture through the staging application and its normal collector. Query by request_id and compare the final stored event fields, timestamp and event count. Search the entire staging log path for the exact fake token. Test that an unauthorized test role cannot query the stored event. Set retention and access through the actual backend configuration.

   Command, configuration or input

   Check: application record → collector record → indexed event → authorized query → unauthorized query denied

   Expected result · ผลที่ควรเห็น

   One intended event survives collection with its required fields, and the fake secret is absent from application, collector and indexed logs.

   If it fails · เมื่อไม่ผ่าน

   If the event is missing, check buffering, parser errors and timestamp normalization before changing the alert. If a collector adds raw payloads, disable that path before rollout.

Recovery and cleanup

The local example changes no persistent logs. Keep the prior logging configuration and restore it if the candidate drops required events, while correcting any raw-secret path before re-enabling collection.

Apply to actual work and known limits

Use an allowlist per event type in the actual application. This demonstrates field minimization and JSON framing; backend authorization, tamper resistance and retention still require their own operational checks.

The included self-contained Python fixture was executed successfully in this Linux environment. External products and production infrastructure were not exercised.

{"event_time_utc": "2026-11-04T06:32:00.020677+00:00", "event_type": "access_denied", "outcome": "denied", "principal_id": "user-17", "request_id": "r1\nFAKE_EVENT"}
PASS: 5 approved fields; fake secret excluded; one physical record