workflow-builder
Graphical interface to build a workflow file
The haddock3-download application of the main branch is published at https://i-vresse-workflow-builder.netlify.app
The builder allows you to create a complex TOML formatted config file based on a set of JSON schemas.
The workflow builder is organized as a monorepo with packages and apps.
Develop
Requires NodeJS and yarn (tested with v3.2.1).
# Install dependencies
yarn
# Run dev servers
yarn dev
Unit tests
Tests (**/*.test.tsx?) written in vitetest can be run with:
yarn test -- run
To run tests with code coverage use
yarn test -- run --coverage
Creates **/coverage/
directory with HTML and LCOV report.
Integration tests
The integration tests (**/integration-tests/**.spec.ts
) are written in playwright.
Before running test ensure browsers are installed with
cd apps/haddock3-download
npx playwright install chromium
Tests can be run with
yarn test:integration
To run a non-headless chromium browser use
yarn test:headed
The browser will pause when a test calls await page.pause()
, so you can investigate current state.
There is a VS code extension to run integration tests inside editor.
Linting
yarn lint
To autofix lint errors use
yarn lint -- --fix
To generate JSON report use
yarn lint -- --report json > eslint.report.json
Build
To build production distribution run
yarn build
Which will create apps/*/dist/
directories which should be hosted on the web somewhere.
The build also creates packages/*/dist
directories which should be published to npmjs.com.
Component development
Components can be developed/tested/documented using storybook.
The storybooks of the main branch are hosted at
Storybook can be started locally with
yarn storybook
Format
Workflow archive
The workflow builder creates a zip file with a workflow configuration file called workflow.cfg
in TOML format.
The configuration file contains paths to input files which are included in the zip file.
Workflow configuration file
The workflow configuration file consists out of 2 parts:
- Global parameters, which are available to engine and each node.
- Tables with parameters for each node the workflow should run.
An uploaded workflow configuration file can contain tables with the same name (this is more lenient then the TOML format).
A generated workflow configuration file with the same node twice will have a TOML string with [somenode]
and ['somenode.2']
table respectively.
Catalog
The catalog is a YAML formatted file which tells the app what nodes are available. In has the following info:
- global: Description of global parameters
- schema: What parameters are valid. Formatted as JSON schema draft 7.
- uiSchema: How the form for filling the parameters should be rendered.
- tomlSchema: How toml keys are mapped to in-memory representation.
- nodes: Description of available nodes.
- id: Identifier of node, for computers
- label: Label of node, for humans
- category: Category to which node belongs
- description: Text describing what node needs, does and produces.
- schema: What parameters are valid. Formatted as JSON schema draft 7.
- uiSchema: How the form for filling the parameters should be rendered.
- tomlSchema: How toml keys are mapped to in-memory representation.
- catagories: Descriptions of node categories
- name: Name of category
- description: Description of category
- collapsed (optional): Whether category should be rendered collapsed initially
- examples: Title and link to example workflows
- map with title as key and link as value
- title: Title of the catalog
- nodeLegend: Legend of the node in the app. Default is 'Node'
schema
See docs/schema.md.
uiSchema
See docs/uiSchema.md.
tomlSchema
See docs/tomlSchema.md.
Catalog index
In the worklfow builder you can pick a catalog from a list. This list gets downloaded from public/catalog/index.json and is formatted like
[
["<title of catalog>", "<URL of catalog YAML file>"]
]
The first catalog in the index.json file will be shown when you open the app.
The haddock3 catalogs can be generated by a Python script in packages/haddock3_catalog from the haddock3 library. The haddock3 catalogs and example are symbolicly linked to /app/*/public
.