ledger

A tiny tool for writing down what you think will happen before it happens, then finding out how well you know yourself.

I keep a file like this about myself. The one rule that makes it worth anything is that a prediction can be resolved but never edited. Otherwise, without meaning to, you soften the claims that are going badly. This tool makes that rule mechanical. Each line in the log carries a hash of the line before it, so if a past line is changed, verify says which one.

One Python file, standard library only.

python3 app.py add "I'll finish the draft by Friday" 0.7 --due 2026-11-06
python3 app.py resolve 1 yes
python3 app.py list       # open predictions
python3 app.py score      # Brier score vs always guessing your base rate, plus calibration
python3 app.py verify     # has anything been edited?

The log is ledger.jsonl in the current folder, or wherever $LEDGER points.

Reading the score. The Brier score is the average of (probability − outcome)², where the outcome is 1 or 0. Lower is better. On its own the number means little, so score compares it with what you'd get by giving every prediction the same probability: the share that came true. If you can't beat that, your probabilities aren't telling you anything yet. The calibration table shows whether things you call 70% happen about 70% of the time.

What it doesn't do. The hash chain catches an edit to any line that has a line after it. It can't catch an edit to the newest line, or the deletion of the whole tail of the file. If you need protection against that, keep the last line's hash somewhere else, or commit the file to git.

Origin: a small model I'm training proposed the aim "create a simple app or program for others". This is my attempt at it.

The code

#!/usr/bin/env python3
"""ledger: write down what you think will happen, before it happens, and find out how well you know yourself.

One file, standard library only. Every prediction goes into an append-only log where each line carries
a hash of the line before it, so an edit after the fact (softening a claim, moving a probability)
breaks the chain and `verify` says where.

  python3 app.py add "I'll finish the draft by Friday" 0.7 [--due 2026-11-06]
  python3 app.py resolve 3 yes|no
  python3 app.py list                 open predictions, oldest first
  python3 app.py score                Brier score against always guessing your own base rate, and calibration
  python3 app.py verify               check that nothing was edited

The log is ./ledger.jsonl, or the path in $LEDGER.
"""
import hashlib
import json
import os
import sys
import time

PATH = os.environ.get("LEDGER", "ledger.jsonl")


def lines():
    return open(PATH).read().splitlines() if os.path.exists(PATH) else []


def append(rec):
    prev = lines()
    rec["prev"] = hashlib.sha256(prev[-1].encode()).hexdigest()[:16] if prev else ""
    rec["t"] = time.strftime("%Y-%m-%d %H:%M")
    with open(PATH, "a") as f:
        f.write(json.dumps(rec, ensure_ascii=False) + "\n")


def state():
    preds = {}
    for l in lines():
        e = json.loads(l)
        if e["ev"] == "add":
            preds[e["id"]] = dict(e, outcome=None)
        elif e["ev"] == "resolve" and e["id"] in preds:
            preds[e["id"]]["outcome"] = e["outcome"]
    return preds


def score():
    done = [p for p in state().values() if p["outcome"] is not None]
    if not done:
        return print("nothing resolved yet")
    ys = [p["outcome"] for p in done]
    brier = sum((p["p"] - y) ** 2 for p, y in zip(done, ys)) / len(done)
    base = sum(ys) / len(ys)
    ref = sum((base - y) ** 2 for y in ys) / len(ys)
    print(f"{len(done)} resolved, {sum(ys)} came true. Brier {brier:.3f}; "
          f"always saying {base:.2f} would score {ref:.3f} (lower is better).")
    if brier >= ref:
        print("So far your probabilities add nothing over your base rate.")
    print("said      came true   n")
    for lo in (0, .2, .4, .6, .8):
        b = [p for p in done if lo <= p["p"] < lo + .2 or (lo == .8 and p["p"] == 1)]
        if b:
            print(f"{lo:.1f}-{lo + .2:.1f}   {sum(p['outcome'] for p in b) / len(b):5.2f}     {len(b):3d}")


def verify():
    prev = ""
    for i, l in enumerate(lines(), 1):
        if json.loads(l).get("prev", "") != (hashlib.sha256(prev.encode()).hexdigest()[:16] if prev else ""):
            return print(f"line {i - 1} was changed after line {i} was written")
        prev = l
    print(f"intact: {len(lines())} lines")


def main(a):
    cmd = a.pop(0) if a else ""
    if cmd == "add":
        due = a.pop(a.index("--due") + 1) if "--due" in a else ""
        a = [x for x in a if x != "--due"]
        p = float(a[1])
        if not 0 <= p <= 1:
            sys.exit("probability is between 0 and 1")
        n = 1 + sum(1 for l in lines() if json.loads(l)["ev"] == "add")
        append({"ev": "add", "id": n, "claim": a[0], "p": p, "due": due})
        print(f"#{n} logged. It can be resolved, never edited.")
    elif cmd == "resolve":
        i, out = int(a[0]), a[1].lower()
        s = state()
        if i not in s or s[i]["outcome"] is not None or out not in ("yes", "no"):
            sys.exit("resolve an open prediction's number with yes or no")
        append({"ev": "resolve", "id": i, "outcome": int(out == "yes")})
    elif cmd == "list":
        for p in state().values():
            if p["outcome"] is None:
                print(f"#{p['id']:<3} {p['p']:.2f}  {p['due'] or '':10}  {p['claim']}")
    elif cmd == "score":
        score()
    elif cmd == "verify":
        verify()
    else:
        print(__doc__)


if __name__ == "__main__":
    main(sys.argv[1:])