OpenMergeAPI

OpenMergeAPI is a language agnostic specification for defining an OpenAPI 3.1 document through filepaths and their "exported" values.

Instead of writing a JSON file like this:

{
	"paths": {
		"/users/{userId}": {
			"get": {
				"summary": "Get User"
			}
		}
	}
}

You might create a TOML file at paths/users/{userId}/get.toml that has this:

summary = "Get User"

Or you might even create a plaintext file at paths/users/{userId}/get/summary.txt that simply has the text Get User.

Note: This specification is language agnostic, and not all languages call the thing a file provides an "export". In the remainder of the documentation, the word "export" simply means "the variable, function, etc that the file defines and makes accessible".

Table of Contents

Problems with Current Solutions

The OpenAPI specifications require you to write in JSON or YAML which are, for various reasons, not very friendly for developers to work with. (Reviewing proposed changes in JSON, for example, is often painful.)

Many language specific frameworks have been created to make the developer experience easier. Typically, these frameworks (systems, tools, etc) are targeted to a specific language, for example adding decorators to classes in TypeScript:

export class CreateCatDto {
	@ApiProperty()
	name: string;
}

One problem with this is portability: if you define everything in Java you can't directly use that in TypeScript, for example.

The other problem is that, quite frequently, the language is not well suited to some particular aspect of the OpenAPI specification. In particular, long-form descriptive text is often shoehorned in:

@ApiProperty({
  description: 'Here is a description that is'
  	+ '*long* enough to need multiple lines.'
  	+ 'Also it\'s annoying to escape characters.',
})

Not only do you have the annoyance of quotes, concatenation (note the missing spaces?), and escaping, you also miss the ability to use natural IDE tools for Markdown syntax.

Objective

There are two primary goals:

  1. We should not keep "re-inventing the wheel" for well designed APIs: definitions and descriptions should be easy to reuse and extend.

  2. Those API definitions should be easy for a human to read and reason about. It should be easy to see what changed in a git diff.

Introduction

In this specification, files in folders map to nodes in an OpenAPI document. For example, you might have a JavaScript file like this:

// paths/hello/get.js
export const summary = 'Says Hello'

Or a TOML file like this:

# paths/hello/get.toml
summary = "Says Hello"

Or even a YAML file, with deeper properties, like this:

# paths/hello.yml
get:
  summary: Says Hello

All three of these would map to this OpenAPI document:

{
	"paths": {
		"/hello": {
			"get": {
				"summary": "Says Hello"
			}
		}
	}
}

You can merge multiple files and folders together, with an ordered merge precedence. For example, given these two YAML files:

# group-1/paths/hello/get.yml
summary: Says Hello

# group-2/paths/hello/get.yml
summary: Hello World

If you merged with group-1 first, you would get this OpenAPI document:

{
	"paths": {
		"/hello": {
			"get": {
				"summary": "Hello World"
			}
		}
	}
}

Whereas if you merged with group-2 first, you would get this slightly different OpenAPI document:

{
	"paths": {
		"/hello": {
			"get": {
				"summary": "Says Hello"
			}
		}
	}
}

File Keypaths

The folder and the file name, minus the file extension, together define a keypath to an object. For example, the file a/b/c.txt would represent a keypath to the property c in this JSON object:

{
	"a": {
		"b": {
			"c": "text"
		}
	}
}

The file name _ is reserved, it simply means to apply the file contents to the folder path only. For example, the file a/b/_.txt would represent a keypath to the property b in the above JSON object.

For example, both info/description.md and info/description/_.md would be equally valid references to the description field in the Info Object.

What Files Provide

The OpenAPI specification says a document must be a valid JSON or YAML document, but this specification is meant to be language agnostic, so we define a valid OpenMergeAPI file as any file which exports (or more loosely "provides", if necessary) a key-value dictionary, where the key is a valid scalar string, and the value is one of two options:

  1. An additional key-value dictionary with the same constraints as the file overall, or
  2. Any of the primitive data types which map in that language to JSON or YAML values.

The exact methodology is left as an implementation detail to the specific language, but here are some examples of what could be reasonable in a few other languages:

JavaScript

// paths/hello/get.js
export const summary = 'Says Hello'
export const externalDocs = {
	// deeper property
	url: 'https://site.com',
}

TOML

# paths/hello/get.toml
summary = "Says Hello"
[externalDocs]
# deeper property
url = "https://site.com"

YAML

# paths/hello/get.yml
summary: Says Hello
externalDocs:
	# deeper property
  url: https://site.com

Abstraction Complexities

This specification is a convenient abstraction, but there are a few things from the OpenAPI specification that do not precisely map to a folder/file based system.

The following items are the known complexities introduced by this abstraction.

Language and Syntax Conflicts

Not all languages can export the keys named in the OpenAPI specifications. For example, the key in is a reserved keyword in JavaScript, making this invalid:

// components/parameters/item_id.js
export const in = 'path'

It is up to the implementations of this specification to handle the translation between language limitations and OpenAPI names, where necessary.

For example, one solution in JavaScript that exports in and is valid looks like this:

// components/parameters/item_id.js
const _in = 'path'
export { _in as in }

However, another approach is for the language implementation to specify that an exported _in will map to the OpenAPI in key:

// components/parameters/item_id.js
export const _in = 'path'

Array Items

Some items in the OpenAPI specification are arrays. For example, at the root the tags property is a list of Tag Objects:

tags:
  - name: cats
    description: Things about felines.
  - name: dogs
    description: Things about canines.

Although you can represent any OpenAPI property as a filepath, arrays are not represented as first class items in most filesystems.

One solution could be to have numeric filenames, e.g. you could have:

# /tags/0.yml
name: cats
description: Things about felines.

# /tags/1.yml
name: dogs
description: Things about canines.

This is an unpleasant developer experience, as the filename 0.yml and 1.yml would have no helpful meaning. You'd have to open the file to have any idea of what was in it.

However, the things which are represented as arrays in the OpenAPI specifications can be grouped into two categories: 1) those which have a required name property, and 2) those that don't have a name property.

This yields the following intuitive approach to arrays:

Name is Required

Both the parameters and tags fields are arrays, and those objects have name as a required property.

Paths:

Tags:

Resolution:

For the OpenMergeAPI specification, if these array items are stored in filepaths, the filepath without the extension is used for the name property of the object.

For example, in the following TOML file the tag name cat is derived from the filepath:

# tags/cat.toml
externalDocs = "https://site.com/docs"
description = "Cats are felines."

You may also export name as a property. An exported name is a higher precedence in the merge order. This can be useful for filesystem behaviour problems.

⚠️ Adding name = "dog" to the above TOML file would make the final merged tag name dog instead of cat. This would likely be very confusing!

Name is Not Present

Both the security and servers keys are arrays, but the respective objects do not have name as a property.

Security:

Servers:

Resolution:

For the OpenMergeAPI specification, these array items may be stored in filepaths. The filepath is used as a key for merge precedence, but otherwise is unused.

For example, given this OpenAPI example:

{
	"security": [
		{
			"petstore_auth":[
				"write:pets",
				"read:pets"
			]
		}
	]
}

It could map to a TOML file like this:

# security/petstore-auth.toml
petstore_auth = [
  "write:pets",
  "read:pets",
]

The filename petstore-auth.toml and the property name petstore_auth are similar, but that similarity does not have programmatic significance.

⚠️ Treat the filename like a function name: it should describe what's inside accurately. Naming it incorrectly would be bad in similar ways to a poorly named function.

File Path Characters

Not every field name in the OpenAPI can be properly represented as filepaths on all file systems.

Case Sensitivity:

The OpenAPI specification says that "all field names in the specification are case sensitive", however many file systems are not case sensitive, leading to problems where the path /users/{userid} is identical to /users/{userId} on some file systems but not on others.

⚠️ Be careful with case sensitivity! Some filesystems (e.g. macOS) show a case distinction in the file explorer, but if you rename a file and only change the casing it will not get noticed by git and other tools.

Reserved Characters/Words:

The rules for OpenAPI templated path strings allow characters that can often be problematic to express on certain file systems. For example some filesystems do not allow characters like : or < and > (angle brackets).

Windows also will not allow for files to be named certain things like CON, PRN, AUX, and so on. Windows also does not allow for files to begin or end with a space character.

Resolution:

For maximum cross-platform consistency, any folder or file name must not use the following "unsafe" characters: /, \, <, >, |, ?, *, ", ', or :.

In addition, any file can export a reserved property named __filename which must be a string with the file extension (if present). This will be used to override the existing filename, allowing for safe use of these unsafe characters.

⚠️ This should not be considered "best practice", as it can lead to confusion. It is specified as an escape hatch to support migration of legacy systems.

During merging, the actual filename is the key to be merged on to, so e.g. a file with an actual name cats.yml that exported __filename: feline.yml would need a corresponding cats.yml to override it, not a feline.yml file.

Path Ambiguity

In the normal OpenAPI specifications, the entire API path is the field name, for example:

{
	"paths": {
		"/config/get": {
			"summary": "Get a Config"
		}
	}
}

However, this specification represents the "/config/get" as a filepath, which could be mapped to this valid filepath and TOML file:

# paths/config/get.toml
summary = "Get a Config"

However, in the OpenAPI specifications, both the Path Item Object and the Operation Object support the summary field, meaning that the TOML filepath could be interpreted as either of these:

Summary for GET /config

{
	"paths": {
		"/config": {
			"get": {
				"summary": "Get a Config"
			}
		}
	}
}

Summary for * /config/get

{
	"paths": {
		"/config/get": {
			"summary": "Get a Config"
		}
	}
}

The depth that this ambiguity is possible is very limited: it only applies to Path Item Objects where the path ends with any of the HTTP methods (e.g. get, put, post, etc.) or the string servers or parameters.

To prevent errors arising from ambiguity, all HTTP paths must

Merging Strategy

You should be able to define a set of things for cats over here and dogs over there and merge them nicely together.

file versus folder

multiple folders merged, what precedence

Functionality NEEDS NEW NAME

Define how to specify "there's a function here" (aka a request handler) ???

that might not need to be part of the specs?