Garmelon/tablejohn
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Reformat everything

Browse source
This commit is contained in:
Joscha 2024-05-11 18:32:36 +02:00
parent 93663fff8c
commit 36ce75b43d
12 changed files with 560 additions and 526 deletions
Download patch file Download diff file Expand all files Collapse all files

142
DESIGN.md
View file

@ -11,57 +11,57 @@ think them through.
- Locally, tablejohn should just work™ without custom config.
- However, some cli args might need to be specified for full functionality.
- The db contains...
- Commits and their relationships
- Branches and whether they're tracked
- Runs and their measurements
- Queue of commits
- Commits and their relationships
- Branches and whether they're tracked
- Runs and their measurements
- Queue of commits
- The in-memory state also contains...
- Connected workers and their state
- From this follows the list of in-progress runs
- Connected workers and their state
- From this follows the list of in-progress runs
- Workers...
- Should be robust
- Noone wants to lose a run a few hours in, for any reason
- Explicitly design for loss of connection, server restarts
- Also design for bench script failures
- Can connect to more than one tablejohn instance
- Use task runtime-based approach to fairness
- Steal tasks based on time already spen on task
- Use plain old http requests (with BASIC auth) to communicate with server
- Store no data permanently
- Should be robust
- Noone wants to lose a run a few hours in, for any reason
- Explicitly design for loss of connection, server restarts
- Also design for bench script failures
- Can connect to more than one tablejohn instance
- Use task runtime-based approach to fairness
- Steal tasks based on time already spen on task
- Use plain old http requests (with BASIC auth) to communicate with server
- Store no data permanently
- Nice-to-have but not critical
- Statically checked links
- Statically checked paths for static files
- Statically checked links
- Statically checked paths for static files
## Web pages
- GET `/`
- Tracked and untracked refs
- Recent significant changes?
- "What's the state of the repo?"
- Tracked and untracked refs
- Recent significant changes?
- "What's the state of the repo?"
- GET `/graph/`
- Interactive graph
- Change scope interactively
- Change metrics interactively
- Interactive graph
- Change scope interactively
- Change metrics interactively
- GET `/queue/`
- List of workers and their state
- List of unfinished runs
- "What's the state of the infrastructure?"
- List of workers and their state
- List of unfinished runs
- "What's the state of the infrastructure?"
- GET `/commit/<hash>/`
- Show details of a commit
- Link to parents, chilren, runs in chronological order
- Resolve refs and branch names to commit hashes -> redirect
- Show details of a commit
- Link to parents, chilren, runs in chronological order
- Resolve refs and branch names to commit hashes -> redirect
- GET `/run/<rid>/`
- Show details of a run
- Link to commit, other runs in chronological order
- Links to compare against previous run, closest tracked ancestors?
- Resolve refs, branch names and commits to their latest runs -> redirect
- Show details of a run
- Link to commit, other runs in chronological order
- Links to compare against previous run, closest tracked ancestors?
- Resolve refs, branch names and commits to their latest runs -> redirect
- GET `/compare/<rid1>/`
- Select/search run to compare against?
- Enter commit hash or run id
- Resolve refs, branch names and commits to their latest runs -> redirect
- Select/search run to compare against?
- Enter commit hash or run id
- Resolve refs, branch names and commits to their latest runs -> redirect
- GET `/compare/rid1/<rid2>/`
- Show changes from rid2 to rid1
- Resolve refs, branch names and commits to their latest runs -> redirect
- Show changes from rid2 to rid1
- Resolve refs, branch names and commits to their latest runs -> redirect
## Worker interaction
@ -76,34 +76,34 @@ This allows more human-readable and permanent links to workers than something
like session ids.
- POST `/api/worker/status`
- Main endpoint for worker/server coordination
- Worker periodically sends current status to server
- Includes a secret randomly chosen by the worker
- Subsequent requests must include exactly the same secret
- Protects against the case where multiple workers share the same name
- Worker may include request for new work
- If so, server may respond with a commit hash and bench method
- Worker may include current work
- If so, server may respond with request to abort the work
- Main endpoint for worker/server coordination
- Worker periodically sends current status to server
- Includes a secret randomly chosen by the worker
- Subsequent requests must include exactly the same secret
- Protects against the case where multiple workers share the same name
- Worker may include request for new work
- If so, server may respond with a commit hash and bench method
- Worker may include current work
- If so, server may respond with request to abort the work
- GET `/api/worker/repo/<hash>/tree.tar.gz`
- Get tar-ed commit from the server's repo, if any exists
- Get tar-ed commit from the server's repo, if any exists
- GET `/api/worker/bench-repo/<hash>/tree.tar.gz`
- Get tar-ed commit from the server's bench repo, if any exist
- Get tar-ed commit from the server's bench repo, if any exist
## CLI Args
tablejohn can be run in one of two modes: Server mode, and worker mode.
- server
- Run a web server that serves the contents of a db
- Optionally, specify repo to update the db from
- Optionally, launch local worker (only if repo is specified)
- When local worker is enabled, it ignores the worker section of the config
- Instead, a worker section is generated from the server config
- This approach should make `--local-worker` more fool-proof
- Run a web server that serves the contents of a db
- Optionally, specify repo to update the db from
- Optionally, launch local worker (only if repo is specified)
- When local worker is enabled, it ignores the worker section of the config
- Instead, a worker section is generated from the server config
- This approach should make `--local-worker` more fool-proof
- worker
- Run only as worker (when using external machine for workers)
- Same config file format as server, just uses different parts
- Run only as worker (when using external machine for workers)
- Same config file format as server, just uses different parts
## Config file and options
@ -111,19 +111,19 @@ Regardless of the mode, the config file is always loaded the same way and has
the same format. It is split into these chunks:
- web (ignored in worker mode)
- Everything to do with the web server
- What address and port to bind on
- What url the site is being served under
- Everything to do with the web server
- What address and port to bind on
- What url the site is being served under
- repo (ignored in worker mode)
- Everything to do with the repo the server is inspecting
- Name (derived from repo path if not specified here)
- How frequently to update the db from the repo
- A remote URL to update the repo from
- Whether to clone the repo if it doesn't yet exist
- Everything to do with the repo the server is inspecting
- Name (derived from repo path if not specified here)
- How frequently to update the db from the repo
- A remote URL to update the repo from
- Whether to clone the repo if it doesn't yet exist
- worker (ignored in server mode)
- Name (uses system name by default)
- Custom bench dir path (creates temporary dir by default)
- List of servers, each of which has...
- Token to authenticate with
- Base url to contact
- Weight to prioritize with (by total run time + overhead?)
- Name (uses system name by default)
- Custom bench dir path (creates temporary dir by default)
- List of servers, each of which has...
- Token to authenticate with
- Base url to contact
- Weight to prioritize with (by total run time + overhead?)