mots - Module Ownership in Tree System

Installation

Mots can be installed using pip:

$ pip install mots

You can check for pre-releases by passing --pre to the pip install command.

Command Line Usage

You can get command line usage help by typing mots --help at the command line.

Initializing a repo

To create an initial mots.yaml file in the current repo, run the following command:

$ mots init

This will create an empty configuration file that looks like this:

repo: test-repo
created_at: '2022-02-11T12:38:57.241494'
updated_at: '2022-02-11T12:38:57.241550'
people: []
modules: []

Adding a Module

To add a new module to your mots configuration, you can either add it directly to the YAML file, or you could use the interactive mots module add command.

$ mots module add
Enter a machine name for the new module (e.g. core_accessibility): example
Enter a human readable name (e.g. Core: Accessibility): Example
Enter a description for the new module: This is an example module.
Enter a comma separated list of owner bugzilla IDs: 633708
Enter a comma separated list of peer bugzilla IDs:
Enter a comma separated list of paths to include: example.text
Enter a comma separated list of paths to exclude:
Enter a machine name of the parent module (optional):

The above code will create a new module in mots.yaml so that the file will look like this:

repo: test-repo
created_at: '2022-02-14T09:08:38.055168'
updated_at: '2022-02-14T09:10:08.096987'
people: []
modules:
  - machine_name: example
    name: Example
    description: This is an example module.
    includes:
      - example.text
    excludes: []
    owners:
      - bmo_id: 633708
    peers: []
    meta:

Note that the only required attribute under “owners” is the bmo_id field. You can optionally add this information manually, and then run the mots clean command which will query the Bugzilla API to fetch the remaining information.

repo: test-repo
created_at: '2022-02-14T09:08:38.055168'
updated_at: '2022-02-14T09:11:32.991309'
people:
  - &zeid
    bmo_id: 633708
    name: Zeid Zabaneh
    info: '[:zeid]'
    nick: zeid
modules:
  - machine_name: example
    name: Example
    description: This is an example module.
    includes:
      - example.text
    excludes: []
    owners:
      - *zeid
    peers: []
    meta:

Adding a Submodule

To add a submodule, follow the same instructions for adding a module, but specify the machine name of the parent module at the input prompt. For example:

$ mots module add
Enter a machine name for the new module (e.g. core_accessibility): example_submodule
Enter a human readable name (e.g. Core: Accessibility): Example Submodule
Enter a description for the new module: This module is a submodule of the "Example" module.
Enter a comma separated list of owner bugzilla IDs:
Enter a comma separated list of peer bugzilla IDs: 633708
Enter a comma separated list of paths to include: example_submodule/**/*
Enter a comma separated list of paths to exclude:
Enter a machine name of the parent module (optional): example
$ mots clean

This will result in a file that looks like this:

repo: test-repo
created_at: '2022-02-14T09:08:38.055168'
updated_at: '2022-02-14T09:32:52.387222'
people:
  - &zeid
    bmo_id: 633708
    name: Zeid Zabaneh
    info: '[:zeid]'
    nick: zeid
modules:
  - machine_name: example
    name: Example
    description: This is an example module.
    includes:
      - example.text
    excludes: []
    owners:
      - *zeid
    peers: []
    meta:
    submodules:
      - machine_name: example_submodule
        name: Example Submodule
        description: This module is a submodule of the "Example" module.
        includes:
          - example_submodule/**/*
        excludes: []
        owners: []
        peers:
          - *zeid
        meta:

Cleaning `mots.yaml`

Use mots clean to automatically sort and synchronize data in the mots.yaml configuration file. This command requires a MOTS_BUGZILLA_API_KEY environment variable to be set, or the key to be defined in your settings. You can do this by running the following commands, replacing the redacted key with an actual Bugzilla API key:

$ mots settings write BUGZILLA_API_KEY ***************************************
$ mots clean

Note

You can generate a Bugzilla API key in your User Preferences page under the API Keys tab.

Validating `mots.yaml`

Validating your modules ensures that you have all the required keys in your configuration file, and that you have unique machine names for all your modules and submodules. Run the following command to do automatic validation:

$ mots validate

Any modules or submodules that have errors in them will be included in any error output from this command.

Querying a File Path

You can determine which module a file path belongs to by using the mots query command. This command takes an arbitrary number of path arguments, and prints out the modules on the screen.

$ mots query example.text example_submodule/test2
example.text:example
example_submodule/test2:example_submodule

Exporting

Using the mots export command, the configuration can be exported in a different format. Currently only reStructuredText is supported. This command will output the result to standard output.

$ mots export > mots.rst

The exported data will look like this:

=======
Modules
=======

Example
~~~~~~
This is an example module.

.. list-table::
        :stub-columns: 1

        * - Owner(s)
          - zeid
        * - Includes
          - example.text

Example Submodule
=================

This module is a submodule of the "Example" module.

.. list-table::
        :stub-columns: 1

        * - Owner(s)
          - zeid
        * - Peer(s)
          - zeid
        * - Includes
          - example_submodule/\*\*/\*

Debugging

To enable debug mode for any command, pass --debug before the command. For example:

$ mots --debug query example.text

Developer Interface

Module

Module operations and utility functions.

class mots.module.Module(machine_name: str, repo_path: str | pathlib.Path, name: str = '', description: str = '', includes: Optional[list[str]] = None, excludes: Optional[list[str]] = None, owners: Optional[list[dict]] = None, peers: Optional[list[dict]] = None, meta: Optional[dict] = None, parent: Optional[mots.module.Module] = None, submodules: Optional[list[dict]] = None, exclude_submodule_paths: bool = True, exclude_module_paths: bool = False)

A top-level module or a submodule.

Parameters

machine_name – a unique, machine-readable name for the module
repo_path – the path of the top-level repository
name – the name of the module
description – the description of the module
includes – a list of paths (glob format) to include
excludes – a list of paths (glob format) to exclude
owners – a list of owners that will own all paths in this module
peers – a list of peers for this module
meta – a dictionary of meta data related to this module
parent – the parent module of this module, if this is a submodule
submodules – a list of submodules of this module
exclude_submodule_paths – when True, paths in submodules are excluded
exclude_module_paths – when True, common paths in other modules are excluded from the directory index

The Module class wraps modules and submodules in a configuration, and provides useful methods for validation, calculating paths, and serialization.

calculate_paths()

Calculate paths based on inclusions and exclusions.

Upon calling this method, excluded paths are parsed using pathlib.Path.rglob and then subtracted from the included paths, which are parsed in the same way.

Return type: set

serialize()

Return a dictionary with relevant module information.

Return type: dict

validate()

Perform validation on module and submodules recursively.

Starting with the current module, ensure that this module includes at least one valid path and a valid name and machine name, and then run the same validation on all submodules.

Parameters: errors – a list of errors to append to
Return type: list

mots.module.ls(modules: list[mots.module.Module])

Print a list of given modules.

Parameters: modules – a list of Module instances

mots.module.show(modules: list[mots.module.Module], module: str)

Show details for a particular module.

Parameters

modules – a list of Module instances
module – the machine_name of a given Module

Raises

ValueError – when no module matches the given machine name

Directory

Directory classes for mots.

class mots.directory.Directory(config: FileConfig)

Path indexer.

When this class is initialized with a FileConfig instance, it parses the configuration and loads all modules into the instance. In order to use the rest of the methods provided by this class, the load method must be called first.

load(full_paths: bool = False)

Populate file path and people indexes.

Parameters: full_paths – when true, loads all repo paths in filesystem into index.

This method should be called any time there are changes in the filesystem. For example, if the directory index is loaded before a patch is applied, then any new files that are added, removed, or moved, may not resolve correctly.

query(*paths: str) → mots.directory.QueryResult

Query given paths and return a list of corresponding modules.

Parameters: paths – a string representing a path within the repo
Return type: mots.directory.QueryResult

reset_config(): Load config and refresh modules, etc…

class mots.directory.People(people, bmo_data: dict)

A people directory searchable by name, email, or BMO ID.

refresh_by_bmo_id(): Refresh index positions of people by their bugzilla ID.

class mots.directory.Person(bmo_id: int, name: str, nick: str): A class representing a person.

class mots.directory.QueryResult(result, rejected): Helper class to simplify query result interpretation.

CLI

This module sets up parsers and maps cli commands to methods.

mots.cli.add(args: argparse.Namespace) → None: Add a new module or submodule to repo configuration file.

mots.cli.check_hashes(args: argparse.Namespace) → None: Check stored hashes against calculated hashes and exit with appropriate code.

mots.cli.ci(args: argparse.Namespace) → None: Perform CI checks or validations.

mots.cli.clean(args: argparse.Namespace) → None: Run clean methods for configuration file and write to disk.

mots.cli.create_parser(): Create parser, subparsers, and arguments.

mots.cli.export(args: argparse.Namespace) → None: Export repo configuration and write to disk, or print to stdout as needed.

mots.cli.init(args: argparse.Namespace) → None: Initialize mots configuration file.

mots.cli.ls(args: argparse.Namespace) → None: List modules.

mots.cli.main(): Run startup commands and redirect to appropriate function.

mots.cli.query(args: argparse.Namespace) → None: Query list of files for module information.

mots.cli.read(args: argparse.Namespace)

Print the value of a specified settings variable.

If no key is provided, prints all available settings variables.

mots.cli.show(args: argparse.Namespace) → None: Show given module details.

mots.cli.validate(args: argparse.Namespace) → None: Validate configuration and show error output if applicable.

mots.cli.version(): Return version information.

mots.cli.write(args: argparse.Namespace): Set a specified settings variable to the provided value.

Config

Configuration classes used to initialize and manage mots in a repo.

class mots.config.FileConfig(path: pathlib.Path = PosixPath('mots.yaml'))

Loader and writer for filesystem based configuration.

check_hashes() → list[str]

Check that the hashes in the config are up to date.

Upon calling this function, the existing configuration is copied and stripped of volatile keys, then a hash is calculated and compared against the old hash.

If there is a mismatch, return non-zero exit code. Otherwise return 0.

init(): Initialize a repo with a config file, if it does not contain it.

load(): Load configuration from file.

write(hashes: Optional[dict] = None): Write configuration to file, and update the timestamp and hashes.

exception mots.config.ValidationError: Thrown when a particular module is not valid.

mots.config.add(new_module: dict, file_config: mots.config.FileConfig, parent: Optional[str] = None, write: bool = True)

Add a new module to the configuration.

Parameters

module – a dictionary containing module parameters
file_config – an instance of FileConfig
parent – the machine name of the parent module if applicable
write – if set to True, writes changes to disk

mots.config.calculate_hashes(config: dict, export: bytes) → tuple[dict, dict]: Calculate a hash of the yaml config file.

mots.config.clean(file_config: mots.config.FileConfig, write: bool = True)

Clean and re-sort configuration file.

Load configuration from disk, sort modules and submodules by machine_name. If there is no valid machine_name, generate one. Reformat yaml content. Calculate and store hashes if needed. Write changes to disk if needed.

Parameters

file_config – an instance of FileConfig
write – if set to True, writes changes to disk.

mots.config.validate(config: dict, repo_path: str) → list[str] | None

Validate the current state of the config file.

Check if top-level dictionary contains required keys
Check if machine names are unique
Instantiate and run validation on each Module instance

Raises: ValidationError – if any validation errors are detected

Utils

Utility helper functions.

mots.utils.generate_machine_readable_name(display_name, keep_case=False): Turn spaces into underscores, and lower the case. Strip all but alphanumerics.

mots.utils.get_list_input(text: str)

Parse comma separated list in user input into a list.

Parameters: test – the text to prompt the user with

mots.utils.mkdir_if_not_exists(path: pathlib.Path): Check if a directory exists, if not, create it.

mots.utils.parse_real_name(real_name): Parse real_name into name and info.

mots.utils.touch_if_not_exists(path: pathlib.Path): Check if a file exists, if not, create it.

Development environment

To set up a local development environment, run the following commands. Replace the python version with the desired version on your local machine.

make dev-env PY=python3.9
make dev

The above commands will set up a local development environment using the provided python version available on your machine, and subsequently install all required packages in that environment.

Generate coverage report

To generate a standard coverage report, run:

make cov

To generate an html coverage report, run:

make cov-html
make serve-cov

Then navigate to your web browser.

Other make commands

Run make to see all available commands.

usage: make <target>

target is one of:
        help          show this message and exit
        build         build the python package and wheels
        clean         remove temporary files
        cov           run coverage check
        cov-html      generate html coverage report
        dev           setup local dev environment by installing required packages, etc.
        dev-env       create a python virtual environment in ./mots-env
        docs          generate documentation
        requirements  regenerate requirements.txt
        serve-cov     simple http server for coverage report
        serve-docs    simple http server for docs
        test          run the test suite

mots - Module Ownership in Tree System

Installation

Command Line Usage

Initializing a repo

Adding a Module

Adding a Submodule

Cleaning mots.yaml

Validating mots.yaml

Querying a File Path

Exporting

Debugging

Developer Interface

Module

Directory

CLI

Config

Utils

Development environment

Generate coverage report

Other make commands

Cleaning `mots.yaml`

Validating `mots.yaml`