Modular Types

edit: whitewind has repeatedly lost sections of this post while I was editing it. I'm done with whitewind (other problems include interfering with common browser shortcuts like ctrl+l (badly), ctrl+tab, being slow, various small things. I'll probably be on hackmd).

This was mostly just a draft/prospective thing though.

? specifically, or perhaps it's just a member value, ie, class Signature { date: DateTime, author: Profile, signing: Hash, signature: RSASig }, and perhaps this implicitly defines a number of type parameters, Signature<date: DateTime = Dynamic, author: Profile = Dynamic...>. which default to a Dynamic variant which means they can't be checked in type expressions or allocation.)

<!-- this wasn't a good point. Interfaces are trivial to add to any structure schema standard, and there isn't a need for generating interfaces/bindings straight from schemas in the pre-agoric regime: right now servers are much harder to 'make' than clients, so it doesn't matter if they all have to write their own bindings, which are all very bespoke anyway - Lacking interfaces is especially bad. Interfaces define member functions, ie, how you can interact with a remote actor, ie, a server. The interface is the first and sometimes last thing a user needs to know about an API. -->

• Where a type extends another (its supertype), it can be used anywhere its supertype is expected, despite having additional components of its own. Extension is especially important in cross-org contexts where a new type often must provide a fallback type (the supertype) for participants who haven't been updated to recognize the extension yet, or where the extension might never become ubiquitous. Any type system can do this. Atproto schemas currently can't. Leaf sort of can as a result of components being analogous to supertypes (any component type that's always paired with another could be described as a subtype of the other, though additional language would be needed to invalidate entities that present an extension type without the supertype it extends, or to ensure that the inheritance structure is always clear even when the recipient is missing some type definitions) and JSON-ld can also sort of do this as a result of typed properties being able to express a component inheritance hierarchy. This is a massive L for atproto.

Before them, Capnp did a great job with its schemas, though very few adopted capnp, for I think mostly dumb reasons (I suspect people saw its read-in-place APIs, which are less ergonomic than more conventional copy APIs, and decided the whole thing was a step down. But having in-place APIs didn't at all preclude the writing of a copy API, but I guess nobody who was a fan of capnp anticipated that the increased parsing speed granted by the slightly more complex in-place APIs wouldn't be enough to make them popular.)

But content-addressed data can't be altered. If you want to change a type, you can define a new one and define translators for the old one. If you want to add a few fields without replacing the type, you can add another component to the entity that has those fields (intersection types) (In languages that lack intersection types, A & B projects down to AWithB { a: A, b: B }. In rust, could also have impl Deref<A> for AWithB and impl Deref<B> for AWithB which would make it basically behave equivalently to an intersection type.) If you use a custom glossary, you can assign ids to these type components that are about as concise as capnp's field numbers. If you don't use your own glossary, each component will be id'd with the content-address of its type.

You may want to see the schema language. I haven't defined one yet. I've been writing types raw in the underlying cross-language cbor format. It's important that the schema language that most people write in is distinct from the schema format. Languages need to be able to evolve over time and no language should reign forever, while the format needs to be stable.

I've been talking like these shortcomings are condemnatory of the other formats, but really maybe really we should just take those formats and improve them a little. But I don't want to, I'd love to just start from scratch, I don't think there are real benefits to interoperating with these systems given that they're all just microblogging communities (some of the most toxic commenters you can get), and their third-party login UX sucks (mastodon requires you to type your username in and afaik is multistage, bsky keeps logging me out), so like, what's the point???

• A composable, cross-language type system, format, and meta-protocol. A language of languages for social computing. A way of saying all that can easily be said to computers about what your networked objects are.
• A downcast operation that allows modular objects to be extended by anyone with additional fields without ever breaking other peoples' code.
• An upcast operation/Any type that allows, eg, arbitrary objects to be attached to posts in user-extensible web apps with enough type information for the computer to automatically fetch a viewer/editor that the community recommends for that object in that context.
• Modular types was made for [modular web], but it's intended for general use in all languages and frameworks whenever they need to communicate over the network.
• You don't have to validate type components you don't use.
• modular types are referenced with content-addresses, which means anyone can create a type, and if others use it, the type definition will stay available forever, while also guaranteeing that components of a type will never have id or member name collisions. To achieve this, we've expanded on the standard IPLD format by introducing a way of representing self-referential structures, which types often require. This is also sometimes needed for serialisation of object graphs [show examples] in high level applications.
• type system features:
• Inheritance
• Parametric types
• variance (eg, List<int> subtypes List<num>, while Function<num> subtypes Function<int>)
• value inputs, eg: for stating a matrix's dimensions as part of its type.
• With a variable number of inputs ("variadic").
• Intersection types
• Sum types (discriminated unions/case classes/enums)
• If you're working in a language or style that can't handle that kind of type complexity, the binding generators will get it out of the way, because type checks can (and usually need to) be done at runtime, so your language doesn't need to understand the types to benefit here. Indeed, currently, no widely used programming language could support all* of the features of the type system, but it seems so far that this is unproblematic, binding generators/codegen/macros have always been able to represent the data pretty smoothly in the host language.
• Canonical representation: Well, we have it, but mainly when you turn extensibility off (ie, stripped types), because it's actually impossible to have both canonicity and extensibility! Extensibility requires that sometimes a deep equals of upcasted objects will be allowed to disagree with a ref equals, due to the presence of extension components. There's also an implied requirement that the recipient shouldn't have to validate a type component they're not interested in, or to access its components, which means those components will be able to have any type tag and any order while still being valid as an intersection type of its supertypes.