mots - Module Ownership in Tree System
Installation
Mots can be installed using pip:
$ pip install mots
You can check for pre-releases by passing --pre
to the pip install
command.
Command Line Usage
You can get command line usage help by typing mots --help
at the command line.
Initializing a repo
To create an initial mots.yaml
file in the current repo, run the following command:
$ mots init
This will create an empty configuration file that looks like this:
repo: test-repo
created_at: '2022-02-11T12:38:57.241494'
updated_at: '2022-02-11T12:38:57.241550'
people: []
modules: []
Adding a Module
To add a new module to your mots configuration, you can either add it directly to the YAML file, or you could use the interactive mots module add
command.
$ mots module add
Enter a machine name for the new module (e.g. core_accessibility): example
Enter a human readable name (e.g. Core: Accessibility): Example
Enter a description for the new module: This is an example module.
Enter a comma separated list of owner bugzilla IDs: 633708
Enter a comma separated list of peer bugzilla IDs:
Enter a comma separated list of paths to include: example.text
Enter a comma separated list of paths to exclude:
Enter a machine name of the parent module (optional):
The above code will create a new module in mots.yaml
so that the file will look like this:
repo: test-repo
created_at: '2022-02-14T09:08:38.055168'
updated_at: '2022-02-14T09:10:08.096987'
people: []
modules:
- machine_name: example
name: Example
description: This is an example module.
includes:
- example.text
excludes: []
owners:
- bmo_id: 633708
peers: []
meta:
Note that the only required attribute under “owners” is the bmo_id
field. You can optionally add this information manually, and then run the mots clean
command which will query the Bugzilla API to fetch the remaining information.
repo: test-repo
created_at: '2022-02-14T09:08:38.055168'
updated_at: '2022-02-14T09:11:32.991309'
people:
- &zeid
bmo_id: 633708
name: Zeid Zabaneh
info: '[:zeid]'
nick: zeid
modules:
- machine_name: example
name: Example
description: This is an example module.
includes:
- example.text
excludes: []
owners:
- *zeid
peers: []
meta:
Adding a Submodule
To add a submodule, follow the same instructions for adding a module, but specify the machine name of the parent module at the input prompt. For example:
$ mots module add
Enter a machine name for the new module (e.g. core_accessibility): example_submodule
Enter a human readable name (e.g. Core: Accessibility): Example Submodule
Enter a description for the new module: This module is a submodule of the "Example" module.
Enter a comma separated list of owner bugzilla IDs:
Enter a comma separated list of peer bugzilla IDs: 633708
Enter a comma separated list of paths to include: example_submodule/**/*
Enter a comma separated list of paths to exclude:
Enter a machine name of the parent module (optional): example
$ mots clean
This will result in a file that looks like this:
repo: test-repo
created_at: '2022-02-14T09:08:38.055168'
updated_at: '2022-02-14T09:32:52.387222'
people:
- &zeid
bmo_id: 633708
name: Zeid Zabaneh
info: '[:zeid]'
nick: zeid
modules:
- machine_name: example
name: Example
description: This is an example module.
includes:
- example.text
excludes: []
owners:
- *zeid
peers: []
meta:
submodules:
- machine_name: example_submodule
name: Example Submodule
description: This module is a submodule of the "Example" module.
includes:
- example_submodule/**/*
excludes: []
owners: []
peers:
- *zeid
meta:
Cleaning mots.yaml
Use mots clean
to automatically sort and synchronize data in the mots.yaml
configuration file. This command requires a MOTS_BUGZILLA_API_KEY
environment variable to be set, or the key to be defined in your settings. You can do this by running the following commands, replacing the redacted key with an actual Bugzilla API key:
$ mots settings write BUGZILLA_API_KEY ***************************************
$ mots clean
Note
You can generate a Bugzilla API key in your User Preferences page under the API Keys tab.
Validating mots.yaml
Validating your modules ensures that you have all the required keys in your configuration file, and that you have unique machine names for all your modules and submodules. Run the following command to do automatic validation:
$ mots validate
Any modules or submodules that have errors in them will be included in any error output from this command.
Querying a File Path
You can determine which module a file path belongs to by using the mots query
command. This command takes an arbitrary number of path arguments, and prints out the modules on the screen.
$ mots query example.text example_submodule/test2
example.text:example
example_submodule/test2:example_submodule
Exporting
Using the mots export
command, the configuration can be exported in a different format. Currently only reStructuredText is supported. This command will output the result to standard output.
$ mots export > mots.rst
The exported data will look like this:
=======
Modules
=======
Example
~~~~~~
This is an example module.
.. list-table::
:stub-columns: 1
* - Owner(s)
- zeid
* - Includes
- example.text
Example Submodule
=================
This module is a submodule of the "Example" module.
.. list-table::
:stub-columns: 1
* - Owner(s)
- zeid
* - Peer(s)
- zeid
* - Includes
- example_submodule/\*\*/\*
Debugging
To enable debug mode for any command, pass --debug
before the command. For example:
$ mots --debug query example.text
Developer Interface
Module
Module operations and utility functions.
- class mots.module.Module(machine_name: str, repo_path: str | pathlib.Path, name: str = '', description: str = '', includes: Optional[list[str]] = None, excludes: Optional[list[str]] = None, owners: Optional[list[dict]] = None, peers: Optional[list[dict]] = None, meta: Optional[dict] = None, parent: Optional[mots.module.Module] = None, submodules: Optional[list[dict]] = None, exclude_submodule_paths: bool = True, exclude_module_paths: bool = False)
A top-level module or a submodule.
- Parameters
machine_name – a unique, machine-readable name for the module
repo_path – the path of the top-level repository
name – the name of the module
description – the description of the module
includes – a list of paths (glob format) to include
excludes – a list of paths (glob format) to exclude
owners – a list of owners that will own all paths in this module
peers – a list of peers for this module
meta – a dictionary of meta data related to this module
parent – the parent module of this module, if this is a submodule
submodules – a list of submodules of this module
exclude_submodule_paths – when True, paths in submodules are excluded
exclude_module_paths – when True, common paths in other modules are excluded from the directory index
The
Module
class wraps modules and submodules in a configuration, and provides useful methods for validation, calculating paths, and serialization.- calculate_paths()
Calculate paths based on inclusions and exclusions.
Upon calling this method, excluded paths are parsed using
pathlib.Path.rglob
and then subtracted from the included paths, which are parsed in the same way.- Return type
set
- serialize()
Return a dictionary with relevant module information.
- Return type
dict
- validate()
Perform validation on module and submodules recursively.
Starting with the current module, ensure that this module includes at least one valid path and a valid name and machine name, and then run the same validation on all submodules.
- Parameters
errors – a list of errors to append to
- Return type
list
- mots.module.ls(modules: list[mots.module.Module])
Print a list of given modules.
- Parameters
modules – a list of
Module
instances
- mots.module.show(modules: list[mots.module.Module], module: str)
Show details for a particular module.
Directory
Directory classes for mots.
- class mots.directory.Directory(config: FileConfig)
Path indexer.
When this class is initialized with a
FileConfig
instance, it parses the configuration and loads all modules into the instance. In order to use the rest of the methods provided by this class, theload
method must be called first.- load(full_paths: bool = False)
Populate file path and people indexes.
- Parameters
full_paths – when true, loads all repo paths in filesystem into index.
This method should be called any time there are changes in the filesystem. For example, if the directory index is loaded before a patch is applied, then any new files that are added, removed, or moved, may not resolve correctly.
- query(*paths: str) mots.directory.QueryResult
Query given paths and return a list of corresponding modules.
- Parameters
paths – a string representing a path within the repo
- Return type
- reset_config()
Load config and refresh modules, etc…
- class mots.directory.People(people, bmo_data: dict)
A people directory searchable by name, email, or BMO ID.
- refresh_by_bmo_id()
Refresh index positions of people by their bugzilla ID.
- class mots.directory.Person(bmo_id: int, name: str, nick: str)
A class representing a person.
- class mots.directory.QueryResult(result, rejected)
Helper class to simplify query result interpretation.
CLI
This module sets up parsers and maps cli commands to methods.
- mots.cli.add(args: argparse.Namespace) None
Add a new module or submodule to repo configuration file.
- mots.cli.check_hashes(args: argparse.Namespace) None
Check stored hashes against calculated hashes and exit with appropriate code.
- mots.cli.ci(args: argparse.Namespace) None
Perform CI checks or validations.
- mots.cli.clean(args: argparse.Namespace) None
Run clean methods for configuration file and write to disk.
- mots.cli.create_parser()
Create parser, subparsers, and arguments.
- mots.cli.export(args: argparse.Namespace) None
Export repo configuration and write to disk, or print to stdout as needed.
- mots.cli.init(args: argparse.Namespace) None
Initialize mots configuration file.
- mots.cli.ls(args: argparse.Namespace) None
List modules.
- mots.cli.main()
Run startup commands and redirect to appropriate function.
- mots.cli.query(args: argparse.Namespace) None
Query list of files for module information.
- mots.cli.read(args: argparse.Namespace)
Print the value of a specified settings variable.
If no key is provided, prints all available settings variables.
- mots.cli.show(args: argparse.Namespace) None
Show given module details.
- mots.cli.validate(args: argparse.Namespace) None
Validate configuration and show error output if applicable.
- mots.cli.version()
Return version information.
- mots.cli.write(args: argparse.Namespace)
Set a specified settings variable to the provided value.
Config
Configuration classes used to initialize and manage mots in a repo.
- class mots.config.FileConfig(path: pathlib.Path = PosixPath('mots.yaml'))
Loader and writer for filesystem based configuration.
- check_hashes() list[str]
Check that the hashes in the config are up to date.
Upon calling this function, the existing configuration is copied and stripped of volatile keys, then a hash is calculated and compared against the old hash.
If there is a mismatch, return non-zero exit code. Otherwise return 0.
- init()
Initialize a repo with a config file, if it does not contain it.
- load()
Load configuration from file.
- write(hashes: Optional[dict] = None)
Write configuration to file, and update the timestamp and hashes.
- exception mots.config.ValidationError
Thrown when a particular module is not valid.
- mots.config.add(new_module: dict, file_config: mots.config.FileConfig, parent: Optional[str] = None, write: bool = True)
Add a new module to the configuration.
- Parameters
module – a dictionary containing module parameters
file_config – an instance of
FileConfig
parent – the machine name of the parent module if applicable
write – if set to True, writes changes to disk
- mots.config.calculate_hashes(config: dict, export: bytes) tuple[dict, dict]
Calculate a hash of the yaml config file.
- mots.config.clean(file_config: mots.config.FileConfig, write: bool = True)
Clean and re-sort configuration file.
Load configuration from disk, sort modules and submodules by machine_name. If there is no valid machine_name, generate one. Reformat yaml content. Calculate and store hashes if needed. Write changes to disk if needed.
- Parameters
file_config – an instance of
FileConfig
write – if set to True, writes changes to disk.
- mots.config.validate(config: dict, repo_path: str) list[str] | None
Validate the current state of the config file.
Check if top-level dictionary contains required keys
Check if machine names are unique
Instantiate and run validation on each
Module
instance
- Raises
ValidationError – if any validation errors are detected
Utils
Utility helper functions.
- mots.utils.generate_machine_readable_name(display_name, keep_case=False)
Turn spaces into underscores, and lower the case. Strip all but alphanumerics.
- mots.utils.get_list_input(text: str)
Parse comma separated list in user input into a list.
- Parameters
test – the text to prompt the user with
- mots.utils.mkdir_if_not_exists(path: pathlib.Path)
Check if a directory exists, if not, create it.
- mots.utils.parse_real_name(real_name)
Parse real_name into name and info.
- mots.utils.touch_if_not_exists(path: pathlib.Path)
Check if a file exists, if not, create it.
Development environment
To set up a local development environment, run the following commands. Replace the python version with the desired version on your local machine.
make dev-env PY=python3.9
make dev
The above commands will set up a local development environment using the provided python version available on your machine, and subsequently install all required packages in that environment.
Generate coverage report
To generate a standard coverage report, run:
make cov
To generate an html coverage report, run:
make cov-html
make serve-cov
Then navigate to your web browser.
Other make commands
Run make to see all available commands.
usage: make <target>
target is one of:
help show this message and exit
build build the python package and wheels
clean remove temporary files
cov run coverage check
cov-html generate html coverage report
dev setup local dev environment by installing required packages, etc.
dev-env create a python virtual environment in ./mots-env
docs generate documentation
requirements regenerate requirements.txt
serve-cov simple http server for coverage report
serve-docs simple http server for docs
test run the test suite