mots - Module Ownership in Tree System

Installation

Mots can be installed using pip:

$ pip install mots

You can check for pre-releases by passing --pre to the pip install command.

Command Line Usage

You can get command line usage help by typing mots --help at the command line.

Initializing a repo

To create an initial mots.yaml file in the current repo, run the following command:

$ mots init

This will create an empty configuration file that looks like this:

repo: test-repo
created_at: '2022-02-11T12:38:57.241494'
updated_at: '2022-02-11T12:38:57.241550'
people: []
modules: []

Adding a Module

To add a new module to your mots configuration, you can either add it directly to the YAML file, or you could use the interactive mots module add command.

$ mots module add
Enter a machine name for the new module (e.g. core_accessibility): example
Enter a human readable name (e.g. Core: Accessibility): Example
Enter a description for the new module: This is an example module.
Enter a comma separated list of owner bugzilla IDs: 633708
Enter a comma separated list of peer bugzilla IDs:
Enter a comma separated list of paths to include: example.text
Enter a comma separated list of paths to exclude:
Enter a machine name of the parent module (optional):

The above code will create a new module in mots.yaml so that the file will look like this:

repo: test-repo
created_at: '2022-02-14T09:08:38.055168'
updated_at: '2022-02-14T09:10:08.096987'
people: []
modules:
  - machine_name: example
    name: Example
    description: This is an example module.
    includes:
      - example.text
    excludes: []
    owners:
      - bmo_id: 633708
    peers: []
    meta:

Note that the only required attribute under “owners” is the bmo_id field. You can optionally add this information manually, and then run the mots clean command which will query the Bugzilla API to fetch the remaining information.

repo: test-repo
created_at: '2022-02-14T09:08:38.055168'
updated_at: '2022-02-14T09:11:32.991309'
people:
  - &zeid
    bmo_id: 633708
    name: Zeid Zabaneh
    info: '[:zeid]'
    nick: zeid
modules:
  - machine_name: example
    name: Example
    description: This is an example module.
    includes:
      - example.text
    excludes: []
    owners:
      - *zeid
    peers: []
    meta:

Adding a Submodule

To add a submodule, follow the same instructions for adding a module, but specify the machine name of the parent module at the input prompt. For example:

$ mots module add
Enter a machine name for the new module (e.g. core_accessibility): example_submodule
Enter a human readable name (e.g. Core: Accessibility): Example Submodule
Enter a description for the new module: This module is a submodule of the "Example" module.
Enter a comma separated list of owner bugzilla IDs:
Enter a comma separated list of peer bugzilla IDs: 633708
Enter a comma separated list of paths to include: example_submodule/**/*
Enter a comma separated list of paths to exclude:
Enter a machine name of the parent module (optional): example
$ mots clean

This will result in a file that looks like this:

repo: test-repo
created_at: '2022-02-14T09:08:38.055168'
updated_at: '2022-02-14T09:32:52.387222'
people:
  - &zeid
    bmo_id: 633708
    name: Zeid Zabaneh
    info: '[:zeid]'
    nick: zeid
modules:
  - machine_name: example
    name: Example
    description: This is an example module.
    includes:
      - example.text
    excludes: []
    owners:
      - *zeid
    peers: []
    meta:
    submodules:
      - machine_name: example_submodule
        name: Example Submodule
        description: This module is a submodule of the "Example" module.
        includes:
          - example_submodule/**/*
        excludes: []
        owners: []
        peers:
          - *zeid
        meta:

Cleaning mots.yaml

Use mots clean to automatically sort and synchronize data in the mots.yaml configuration file. This command requires a MOTS_BUGZILLA_API_KEY environment variable to be set, or the key to be defined in your settings. You can do this by running the following commands, replacing the redacted key with an actual Bugzilla API key:

$ mots settings write BUGZILLA_API_KEY ***************************************
$ mots clean

Note

You can generate a Bugzilla API key in your User Preferences page under the API Keys tab.

Validating mots.yaml

Validating your modules ensures that you have all the required keys in your configuration file, and that you have unique machine names for all your modules and submodules. Run the following command to do automatic validation:

$ mots validate

Any modules or submodules that have errors in them will be included in any error output from this command.

Querying a File Path

You can determine which module a file path belongs to by using the mots query command. This command takes an arbitrary number of path arguments, and prints out the modules on the screen.

$ mots query example.text example_submodule/test2
example.text:example
example_submodule/test2:example_submodule

Exporting

Using the mots export command, the configuration can be exported in a different format. Currently only reStructuredText is supported. This command will output the result to standard output.

$ mots export > mots.rst

The exported data will look like this:

=======
Modules
=======

Example
~~~~~~
This is an example module.

.. list-table::
        :stub-columns: 1

        * - Owner(s)
          - zeid
        * - Includes
          - example.text

Example Submodule
=================

This module is a submodule of the "Example" module.

.. list-table::
        :stub-columns: 1

        * - Owner(s)
          - zeid
        * - Peer(s)
          - zeid
        * - Includes
          - example_submodule/\*\*/\*

Debugging

To enable debug mode for any command, pass --debug before the command. For example:

$ mots --debug query example.text

Developer Interface

Module

Module operations and utility functions.

class mots.module.Module(machine_name: str, repo_path: str | pathlib.Path, name: str = '', description: str = '', includes: Optional[list[str]] = None, excludes: Optional[list[str]] = None, owners: Optional[list[dict]] = None, peers: Optional[list[dict]] = None, meta: Optional[dict] = None, parent: Optional[mots.module.Module] = None, submodules: Optional[list[dict]] = None, exclude_submodule_paths: bool = True, exclude_module_paths: bool = False)

A top-level module or a submodule.

Parameters
  • machine_name – a unique, machine-readable name for the module

  • repo_path – the path of the top-level repository

  • name – the name of the module

  • description – the description of the module

  • includes – a list of paths (glob format) to include

  • excludes – a list of paths (glob format) to exclude

  • owners – a list of owners that will own all paths in this module

  • peers – a list of peers for this module

  • meta – a dictionary of meta data related to this module

  • parent – the parent module of this module, if this is a submodule

  • submodules – a list of submodules of this module

  • exclude_submodule_paths – when True, paths in submodules are excluded

  • exclude_module_paths – when True, common paths in other modules are excluded from the directory index

The Module class wraps modules and submodules in a configuration, and provides useful methods for validation, calculating paths, and serialization.

calculate_paths()

Calculate paths based on inclusions and exclusions.

Upon calling this method, excluded paths are parsed using pathlib.Path.rglob and then subtracted from the included paths, which are parsed in the same way.

Return type

set

serialize()

Return a dictionary with relevant module information.

Return type

dict

validate()

Perform validation on module and submodules recursively.

Starting with the current module, ensure that this module includes at least one valid path and a valid name and machine name, and then run the same validation on all submodules.

Parameters

errors – a list of errors to append to

Return type

list

mots.module.ls(modules: list[mots.module.Module])

Print a list of given modules.

Parameters

modules – a list of Module instances

mots.module.show(modules: list[mots.module.Module], module: str)

Show details for a particular module.

Parameters
  • modules – a list of Module instances

  • module – the machine_name of a given Module

Raises

ValueError – when no module matches the given machine name

Directory

Directory classes for mots.

class mots.directory.Directory(config: FileConfig)

Path indexer.

When this class is initialized with a FileConfig instance, it parses the configuration and loads all modules into the instance. In order to use the rest of the methods provided by this class, the load method must be called first.

load(full_paths: bool = False)

Populate file path and people indexes.

Parameters

full_paths – when true, loads all repo paths in filesystem into index.

This method should be called any time there are changes in the filesystem. For example, if the directory index is loaded before a patch is applied, then any new files that are added, removed, or moved, may not resolve correctly.

query(*paths: str) mots.directory.QueryResult

Query given paths and return a list of corresponding modules.

Parameters

paths – a string representing a path within the repo

Return type

mots.directory.QueryResult

reset_config()

Load config and refresh modules, etc…

class mots.directory.People(people, bmo_data: dict)

A people directory searchable by name, email, or BMO ID.

refresh_by_bmo_id()

Refresh index positions of people by their bugzilla ID.

class mots.directory.Person(bmo_id: int, name: str, nick: str)

A class representing a person.

class mots.directory.QueryResult(result, rejected)

Helper class to simplify query result interpretation.

CLI

This module sets up parsers and maps cli commands to methods.

mots.cli.add(args: argparse.Namespace) None

Add a new module or submodule to repo configuration file.

mots.cli.check_hashes(args: argparse.Namespace) None

Check stored hashes against calculated hashes and exit with appropriate code.

mots.cli.ci(args: argparse.Namespace) None

Perform CI checks or validations.

mots.cli.clean(args: argparse.Namespace) None

Run clean methods for configuration file and write to disk.

mots.cli.create_parser()

Create parser, subparsers, and arguments.

mots.cli.export(args: argparse.Namespace) None

Export repo configuration and write to disk, or print to stdout as needed.

mots.cli.init(args: argparse.Namespace) None

Initialize mots configuration file.

mots.cli.ls(args: argparse.Namespace) None

List modules.

mots.cli.main()

Run startup commands and redirect to appropriate function.

mots.cli.query(args: argparse.Namespace) None

Query list of files for module information.

mots.cli.read(args: argparse.Namespace)

Print the value of a specified settings variable.

If no key is provided, prints all available settings variables.

mots.cli.show(args: argparse.Namespace) None

Show given module details.

mots.cli.validate(args: argparse.Namespace) None

Validate configuration and show error output if applicable.

mots.cli.version()

Return version information.

mots.cli.write(args: argparse.Namespace)

Set a specified settings variable to the provided value.

Config

Configuration classes used to initialize and manage mots in a repo.

class mots.config.FileConfig(path: pathlib.Path = PosixPath('mots.yaml'))

Loader and writer for filesystem based configuration.

check_hashes() list[str]

Check that the hashes in the config are up to date.

Upon calling this function, the existing configuration is copied and stripped of volatile keys, then a hash is calculated and compared against the old hash.

If there is a mismatch, return non-zero exit code. Otherwise return 0.

init()

Initialize a repo with a config file, if it does not contain it.

load()

Load configuration from file.

write(hashes: Optional[dict] = None)

Write configuration to file, and update the timestamp and hashes.

exception mots.config.ValidationError

Thrown when a particular module is not valid.

mots.config.add(new_module: dict, file_config: mots.config.FileConfig, parent: Optional[str] = None, write: bool = True)

Add a new module to the configuration.

Parameters
  • module – a dictionary containing module parameters

  • file_config – an instance of FileConfig

  • parent – the machine name of the parent module if applicable

  • write – if set to True, writes changes to disk

mots.config.calculate_hashes(config: dict, export: bytes) tuple[dict, dict]

Calculate a hash of the yaml config file.

mots.config.clean(file_config: mots.config.FileConfig, write: bool = True)

Clean and re-sort configuration file.

Load configuration from disk, sort modules and submodules by machine_name. If there is no valid machine_name, generate one. Reformat yaml content. Calculate and store hashes if needed. Write changes to disk if needed.

Parameters
  • file_config – an instance of FileConfig

  • write – if set to True, writes changes to disk.

mots.config.validate(config: dict, repo_path: str) list[str] | None

Validate the current state of the config file.

  • Check if top-level dictionary contains required keys

  • Check if machine names are unique

  • Instantiate and run validation on each Module instance

Raises

ValidationError – if any validation errors are detected

Utils

Utility helper functions.

mots.utils.generate_machine_readable_name(display_name, keep_case=False)

Turn spaces into underscores, and lower the case. Strip all but alphanumerics.

mots.utils.get_list_input(text: str)

Parse comma separated list in user input into a list.

Parameters

test – the text to prompt the user with

mots.utils.mkdir_if_not_exists(path: pathlib.Path)

Check if a directory exists, if not, create it.

mots.utils.parse_real_name(real_name)

Parse real_name into name and info.

mots.utils.touch_if_not_exists(path: pathlib.Path)

Check if a file exists, if not, create it.

Development environment

To set up a local development environment, run the following commands. Replace the python version with the desired version on your local machine.

make dev-env PY=python3.9
make dev

The above commands will set up a local development environment using the provided python version available on your machine, and subsequently install all required packages in that environment.

Generate coverage report

To generate a standard coverage report, run:

make cov

To generate an html coverage report, run:

make cov-html
make serve-cov

Then navigate to your web browser.

Other make commands

Run make to see all available commands.

usage: make <target>

target is one of:
        help          show this message and exit
        build         build the python package and wheels
        clean         remove temporary files
        cov           run coverage check
        cov-html      generate html coverage report
        dev           setup local dev environment by installing required packages, etc.
        dev-env       create a python virtual environment in ./mots-env
        docs          generate documentation
        requirements  regenerate requirements.txt
        serve-cov     simple http server for coverage report
        serve-docs    simple http server for docs
        test          run the test suite