Refactor Mako actions, fix setup-git command & add exec-ryujinx-tasks command (#3)

* Create multiple actions to make Mako easier to use

* Add smoke tests for the new actions

* Check if the required env vars aren't empty

* Fix working directory for execute-command

* Fix action_path references

* Fix broken setup_git command

* Add exec-ryujinx-tasks subcommand

* Ensure python and pipx are installed

* Improve help output

* Add required environment variables to README.md

* Add small script to generate subcommand sections automatically

* Adjust help messages for ryujinx tasks as well

* Remove required argument for positionals

* Add exec-ryujinx-tasks to subcommands list

* Apply black formatting

* Fix event name for update-reviewers
This commit is contained in:
TSRBerry 2024-01-27 20:49:49 +01:00 committed by GitHub
parent 09cd87917d
commit 66a1029bd5
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
15 changed files with 418 additions and 67 deletions

View file

@ -0,0 +1,20 @@
# execute-command
A small composite action to run the specified Mako subcommand.
## Usage
Add the following step to your workflow:
```yml
- name: Execute Ryujinx-Mako command
uses: Ryujinx/Ryujinx-Mako/.github/actions/execute-command@master
with:
command: "<a valid subcommand for Mako>"
args: "<subcommand args>"
app_id: ${{ secrets.MAKO_APP_ID }}
private_key: ${{ secrets.MAKO_PRIVATE_KEY }}
installation_id: ${{ secrets.MAKO_INSTALLATION_ID }}
```

View file

@ -0,0 +1,35 @@
name: 'Mako command'
description: 'Execute a Mako subcommand'
inputs:
command:
description: 'Subcommand to execute with Mako'
required: true
args:
description: 'Arguments for the specified subcommand'
required: true
default: ''
app_id:
description: 'GitHub App ID'
required: true
private_key:
description: 'Private key for the GitHub App'
required: true
installation_id:
description: 'GitHub App Installation ID'
required: true
runs:
using: 'composite'
steps:
- name: Get Mako path
id: path
run: |
echo "mako=$(realpath '${{ github.action_path }}/../../../')" >> $GITHUB_OUTPUT
shell: bash
- run: |
poetry -n -C "${{ steps.path.outputs.mako }}" run ryujinx-mako ${{ inputs.command }} ${{ inputs.args }}
shell: bash
env:
MAKO_APP_ID: ${{ inputs.app_id }}
MAKO_PRIVATE_KEY: ${{ inputs.private_key }}
MAKO_INSTALLATION_ID: ${{ inputs.installation_id }}

View file

@ -6,18 +6,11 @@ It installs poetry and all module dependencies.
## Usage ## Usage
Add the following steps to your workflow: Add the following step to your workflow:
```yml ```yml
- name: Checkout Ryujinx-Mako
uses: actions/checkout@v3
with:
repository: Ryujinx/Ryujinx-Mako
ref: master
path: ".ryujinx-mako"
- name: Setup Ryujinx-Mako - name: Setup Ryujinx-Mako
uses: .ryujinx-mako/.github/actions/setup-mako uses: Ryujinx/Ryujinx-Mako/.github/actions/setup-mako@master
``` ```

View file

@ -1,12 +1,32 @@
name: 'Setup Ryujinx-Mako' name: 'Setup Mako'
description: 'Setup the environment for Ryujinx-Mako' description: 'Setup the environment for Mako'
runs: runs:
using: 'composite' using: 'composite'
steps: steps:
- run: pipx install poetry - name: Get Mako path
id: path
run: |
echo "mako=$(realpath '${{ github.action_path }}/../../../')" >> $GITHUB_OUTPUT
shell: bash
- uses: actions/setup-python@v4
with:
cache: 'poetry'
- name: Ensure pipx is available
run: |
if ! command -v pipx > /dev/null 2>&1; then
echo "$HOME/.local/bin" >> $GITHUB_PATH
python3 -m pip install --user pipx
python3 -m pipx ensurepath
fi
shell: bash
- name: Install poetry
run: pipx install poetry
shell: bash shell: bash
- run: | - run: |
cd .ryujinx-mako cd "${{ steps.path.outputs.mako }}"
poetry install --only main poetry install --only main
shell: bash shell: bash

35
.github/workflows/test.yml vendored Normal file
View file

@ -0,0 +1,35 @@
name: Test
on:
push:
workflow_dispatch:
jobs:
action:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Test Ryujinx-Mako (setup-git)
uses: ./
with:
command: setup-git
app_id: ${{ secrets.MAKO_APP_ID }}
private_key: ${{ secrets.MAKO_PRIVATE_KEY }}
installation_id: ${{ secrets.MAKO_INSTALLATION_ID }}
subactions:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Test setup-mako
uses: ./.github/actions/setup-mako
- name: Test execute-command (setup-git)
uses: ./.github/actions/execute-command
with:
command: setup-git
app_id: ${{ secrets.MAKO_APP_ID }}
private_key: ${{ secrets.MAKO_PRIVATE_KEY }}
installation_id: ${{ secrets.MAKO_INSTALLATION_ID }}

View file

@ -4,38 +4,29 @@ A custom GitHub App to aid Ryujinx with project management and moderation
## Usage ## Usage
1. Add the following steps to your workflow: Add the following step to your workflow:
```yml ```yml
- name: Checkout Ryujinx-Mako - name: Run Ryujinx-Mako
uses: actions/checkout@v3 uses: Ryujinx/Ryujinx-Mako@master
with: with:
repository: Ryujinx/Ryujinx-Mako command: <Mako subcommand>
ref: master args: <subcommand args>
path: '.ryujinx-mako' app_id: ${{ secrets.MAKO_APP_ID }}
private_key: ${{ secrets.MAKO_PRIVATE_KEY }}
- name: Setup Ryujinx-Mako installation_id: ${{ secrets.MAKO_INSTALLATION_ID }}
uses: ./.ryujinx-mako/.github/actions/setup-mako ```
```
## Required environment variables
2. Execute the available commands like this:
- `MAKO_APP_ID`: the GitHub App ID
```yml - `MAKO_PRIVATE_KEY`: the contents of the GitHub App private key
- name: Setup git identity for Ryujinx-Mako - `MAKO_INSTALLATION_ID`: the GitHub App installation ID
run: |
# poetry -n -C .ryujinx-mako run ryujinx-mako <command> [<args>]
# for example:
poetry -n -C .ryujinx-mako run ryujinx-mako setup-git
env:
MAKO_APP_ID: ${{ secrets.MAKO_APP_ID }}
MAKO_PRIVATE_KEY: ${{ secrets.MAKO_PRIVATE_KEY }}
MAKO_INSTALLATION_ID: ${{ secrets.MAKO_INSTALLATION_ID }}
```
## Available commands ## Available commands
``` ```
usage: ryujinx_mako [-h] {setup-git,update-reviewers} ... usage: ryujinx_mako [-h] {setup-git,update-reviewers,exec-ryujinx-tasks} ...
A python module to aid Ryujinx with project management and moderation A python module to aid Ryujinx with project management and moderation
@ -43,9 +34,10 @@ options:
-h, --help show this help message and exit -h, --help show this help message and exit
subcommands: subcommands:
setup-git Set git identity to Ryujinx-Mako {setup-git,update-reviewers,exec-ryujinx-tasks}
setup-git Configure git identity for Ryujinx-Mako
update-reviewers Update reviewers for the specified PR update-reviewers Update reviewers for the specified PR
exec-ryujinx-tasks Execute all Ryujinx tasks for a specific event
``` ```
### setup-git ### setup-git
@ -53,11 +45,11 @@ subcommands:
``` ```
usage: ryujinx_mako setup-git [-h] [-l] usage: ryujinx_mako setup-git [-h] [-l]
Set git identity to Ryujinx-Mako Configure git identity for Ryujinx-Mako
options: options:
-h, --help show this help message and exit -h, --help show this help message and exit
-l, --local Set git identity only for the current repository. -l, --local configure the git identity only for the current repository
``` ```
### update-reviewers ### update-reviewers
@ -68,10 +60,35 @@ usage: ryujinx_mako update-reviewers [-h] repo_path pr_number config_path
Update reviewers for the specified PR Update reviewers for the specified PR
positional arguments: positional arguments:
repo_path repo_path full name of the GitHub repository (format: OWNER/REPO)
pr_number pr_number the number of the pull request to check
config_path config_path the path to the reviewers config file
options: options:
-h, --help show this help message and exit -h, --help show this help message and exit
``` ```
### exec-ryujinx-tasks
```
usage: ryujinx_mako exec-ryujinx-tasks [-h] --event-name EVENT_NAME
--event-path EVENT_PATH [-w WORKSPACE]
repo_path run_id
Execute all Ryujinx tasks for a specific event
positional arguments:
repo_path full name of the GitHub repository (format:
OWNER/REPO)
run_id The unique identifier of the workflow run
options:
-h, --help show this help message and exit
--event-name EVENT_NAME
the name of the event that triggered the workflow run
--event-path EVENT_PATH
the path to the file on the runner that contains the
full event webhook payload
-w WORKSPACE, --workspace WORKSPACE
the working directory on the runner
```

47
action.yml Normal file
View file

@ -0,0 +1,47 @@
name: 'Run Ryujinx-Mako'
description: 'Setup Mako and execute the specified subcommand'
inputs:
command:
description: 'Subcommand to execute with Mako'
required: true
args:
description: 'Arguments for the specified subcommand'
required: true
default: ''
app_id:
description: 'GitHub App ID'
required: true
private_key:
description: 'Private key for the GitHub App'
required: true
installation_id:
description: 'GitHub App Installation ID'
required: true
runs:
using: 'composite'
steps:
- name: Check if Mako was already setup
id: check_dest
run: |
[ -f "${{ github.action_path }}/.ryujinx-mako_setup-done" ] \
&& echo "exists=true" >> $GITHUB_OUTPUT \
|| echo "exists=false" >> $GITHUB_OUTPUT
shell: bash
- name: Setup Mako
if: steps.check_dest.outputs.exists == 'false'
uses: ./.github/actions/setup-mako
- name: Create setup finished flag
if: steps.check_dest.outputs.exists == 'false'
run: touch "${{ github.action_path }}/.ryujinx-mako_setup-done"
shell: bash
- name: Run Mako subcommand
uses: ./.github/actions/execute-command
with:
command: ${{ inputs.command }}
args: ${{ inputs.args }}
app_id: ${{ inputs.app_id }}
private_key: ${{ inputs.private_key }}
installation_id: ${{ inputs.installation_id }}

View file

@ -3,6 +3,7 @@ import logging
from ryujinx_mako import commands from ryujinx_mako import commands
from ryujinx_mako._const import SCRIPT_NAME from ryujinx_mako._const import SCRIPT_NAME
from ryujinx_mako.commands import Subcommand
parser = argparse.ArgumentParser( parser = argparse.ArgumentParser(
prog=SCRIPT_NAME, prog=SCRIPT_NAME,
@ -19,10 +20,11 @@ for subcommand in commands.SUBCOMMANDS:
subcommand_parser = subparsers.add_parser( subcommand_parser = subparsers.add_parser(
subcommand.name(), subcommand.name(),
description=subcommand.description(), description=subcommand.description(),
add_help=True, help=subcommand.description(),
) )
Subcommand.add_subcommand(subcommand.name(), subcommand(subcommand_parser))
# Keep a reference to the subcommand # Keep a reference to the subcommand
subcommands.append(subcommand(subcommand_parser)) subcommands.append(Subcommand.get_subcommand(subcommand.name()))
def run(): def run():

View file

@ -11,6 +11,7 @@ except ImportError:
class ConfigKey(StrEnum): class ConfigKey(StrEnum):
DryRun = "MAKO_DRY_RUN"
AppID = "MAKO_APP_ID" AppID = "MAKO_APP_ID"
PrivateKey = "MAKO_PRIVATE_KEY" PrivateKey = "MAKO_PRIVATE_KEY"
InstallationID = "MAKO_INSTALLATION_ID" InstallationID = "MAKO_INSTALLATION_ID"
@ -19,14 +20,23 @@ class ConfigKey(StrEnum):
NAME = "Ryujinx-Mako" NAME = "Ryujinx-Mako"
SCRIPT_NAME = NAME.lower().replace("-", "_") SCRIPT_NAME = NAME.lower().replace("-", "_")
# Check environment variables if ConfigKey.DryRun not in os.environ.keys() or len(os.environ[ConfigKey.DryRun]) == 0:
for key in ConfigKey: IS_DRY_RUN = False
if key not in os.environ.keys(): # Check environment variables
raise KeyError(f"Required environment variable not set: {key}") for key in ConfigKey:
if key == ConfigKey.DryRun:
continue
if key not in os.environ.keys() or len(os.environ[key]) == 0:
raise KeyError(f"Required environment variable not set: {key}")
APP_ID = int(os.environ[ConfigKey.AppID]) APP_ID = int(os.environ[ConfigKey.AppID])
PRIVATE_KEY = os.environ[ConfigKey.PrivateKey] PRIVATE_KEY = os.environ[ConfigKey.PrivateKey]
INSTALLATION_ID = int(os.environ[ConfigKey.InstallationID]) INSTALLATION_ID = int(os.environ[ConfigKey.InstallationID])
else:
IS_DRY_RUN = True
APP_ID = 0
PRIVATE_KEY = ""
INSTALLATION_ID = 0
GH_BOT_SUFFIX = "[bot]" GH_BOT_SUFFIX = "[bot]"
GH_EMAIL_TEMPLATE = "{user_id}+{username}@users.noreply.github.com" GH_EMAIL_TEMPLATE = "{user_id}+{username}@users.noreply.github.com"

View file

@ -1,12 +1,14 @@
from typing import Type from typing import Type
from ryujinx_mako.commands._subcommand import Subcommand from ryujinx_mako.commands._subcommand import Subcommand
from ryujinx_mako.commands.exec_ryujinx_tasks import ExecRyujinxTasks
from ryujinx_mako.commands.setup_git import SetupGit from ryujinx_mako.commands.setup_git import SetupGit
from ryujinx_mako.commands.update_reviewers import UpdateReviewers from ryujinx_mako.commands.update_reviewers import UpdateReviewers
SUBCOMMANDS: list[Type[Subcommand]] = [ SUBCOMMANDS: list[Type[Subcommand]] = [
SetupGit, SetupGit,
UpdateReviewers, UpdateReviewers,
ExecRyujinxTasks,
] ]
__all__ = SUBCOMMANDS __all__ = SUBCOMMANDS

View file

@ -1,14 +1,23 @@
import logging import logging
from abc import ABC, abstractmethod from abc import ABC, abstractmethod
from argparse import ArgumentParser, Namespace from argparse import ArgumentParser, Namespace
from typing import Any
from github import Github from github import Github
from github.Auth import AppAuth from github.Auth import AppAuth
from ryujinx_mako._const import APP_ID, PRIVATE_KEY, INSTALLATION_ID, SCRIPT_NAME from ryujinx_mako._const import (
APP_ID,
PRIVATE_KEY,
INSTALLATION_ID,
SCRIPT_NAME,
IS_DRY_RUN,
)
class Subcommand(ABC): class Subcommand(ABC):
_subcommands: dict[str, Any] = {}
@abstractmethod @abstractmethod
def __init__(self, parser: ArgumentParser): def __init__(self, parser: ArgumentParser):
parser.set_defaults(func=self.run) parser.set_defaults(func=self.run)
@ -33,10 +42,22 @@ class Subcommand(ABC):
def description() -> str: def description() -> str:
raise NotImplementedError() raise NotImplementedError()
@classmethod
def get_subcommand(cls, name: str):
return cls._subcommands[name]
@classmethod
def add_subcommand(cls, name: str, subcommand):
if name in cls._subcommands.keys():
raise ValueError(f"Key '{name}' already exists in {cls}._subcommands")
cls._subcommands[name] = subcommand
class GithubSubcommand(Subcommand, ABC): class GithubSubcommand(Subcommand, ABC):
_github = Github( _github = (
auth=AppAuth(APP_ID, PRIVATE_KEY).get_installation_auth(INSTALLATION_ID) Github(auth=AppAuth(APP_ID, PRIVATE_KEY).get_installation_auth(INSTALLATION_ID))
if not IS_DRY_RUN
else None
) )
@property @property

View file

@ -0,0 +1,86 @@
import json
import os
from argparse import ArgumentParser, Namespace
from pathlib import Path
from typing import Any
from github.Repository import Repository
from github.WorkflowRun import WorkflowRun
from ryujinx_mako.commands._subcommand import GithubSubcommand
class ExecRyujinxTasks(GithubSubcommand):
@staticmethod
def name() -> str:
return "exec-ryujinx-tasks"
@staticmethod
def description() -> str:
return "Execute all Ryujinx tasks for a specific event"
# noinspection PyTypeChecker
def __init__(self, parser: ArgumentParser):
self._workspace: Path = None
self._repo: Repository = None
self._workflow_run: WorkflowRun = None
self._event: dict[str, Any] = None
self._event_name: str = None
parser.add_argument(
"--event-name",
type=str,
required=True,
help="the name of the event that triggered the workflow run",
)
parser.add_argument(
"--event-path",
type=str,
required=True,
help="the path to the file on the runner that contains the full "
"event webhook payload",
)
parser.add_argument(
"-w",
"--workspace",
type=Path,
required=False,
default=Path(os.getcwd()),
help="the working directory on the runner",
)
parser.add_argument(
"repo_path",
type=str,
help="full name of the GitHub repository (format: OWNER/REPO)",
)
parser.add_argument(
"run_id",
type=int,
help="The unique identifier of the workflow run",
)
super().__init__(parser)
def update_reviewers(self):
# Prepare update-reviewers
self.logger.info("Task: update-reviewers")
args = Namespace()
args.repo_path = self._repo.full_name
args.pr_number = self._event["number"]
args.config_path = Path(self._workspace, ".github", "reviewers.yml")
# Run task
self.get_subcommand("update-reviewers").run(args)
def run(self, args: Namespace):
self.logger.info("Executing Ryujinx tasks...")
self._workspace = args.workspace
self._repo = self.github.get_repo(args.repo_path)
self._workflow_run = self._repo.get_workflow_run(args.run_id)
self._event_name = args.event_name
with open(args.event_path, "r") as file:
self._event = json.load(file)
if args.event_name == "pull_request_target":
self.update_reviewers()
self.logger.info("Finished executing Ryujinx tasks!")

View file

@ -12,14 +12,14 @@ class SetupGit(GithubSubcommand):
@staticmethod @staticmethod
def description() -> str: def description() -> str:
return f"Set git identity to {NAME}" return f"Configure git identity for {NAME}"
def __init__(self, parser: ArgumentParser): def __init__(self, parser: ArgumentParser):
parser.add_argument( parser.add_argument(
"-l", "-l",
"--local", "--local",
action="store_true", action="store_true",
help="Set git identity only for the current repository.", help="configure the git identity only for the current repository",
) )
super().__init__(parser) super().__init__(parser)
@ -29,7 +29,7 @@ class SetupGit(GithubSubcommand):
self.logger.debug(f"Getting GitHub user for: {gh_username}") self.logger.debug(f"Getting GitHub user for: {gh_username}")
user = self.github.get_user(gh_username) user = self.github.get_user(gh_username)
email = GH_EMAIL_TEMPLATE.format(user_id=user.id, username=user.name) email = GH_EMAIL_TEMPLATE.format(user_id=user.id, username=user.login)
if args.local: if args.local:
self.logger.debug("Setting git identity for local repo...") self.logger.debug("Setting git identity for local repo...")
@ -37,7 +37,7 @@ class SetupGit(GithubSubcommand):
self.logger.debug("Setting git identity globally...") self.logger.debug("Setting git identity globally...")
base_command.append("--global") base_command.append("--global")
config = {"user.name": user.name, "user.email": email} config = {"user.name": user.login, "user.email": email}
for option, value in config.items(): for option, value in config.items():
self.logger.info(f"Setting git {option} to: {value}") self.logger.info(f"Setting git {option} to: {value}")
command = base_command.copy() command = base_command.copy()

View file

@ -21,9 +21,19 @@ class UpdateReviewers(GithubSubcommand):
self._reviewers = set() self._reviewers = set()
self._team_reviewers = set() self._team_reviewers = set()
parser.add_argument("repo_path", type=str) parser.add_argument(
parser.add_argument("pr_number", type=int) "repo_path",
parser.add_argument("config_path", type=Path) type=str,
help="full name of the GitHub repository (format: OWNER/REPO)",
)
parser.add_argument(
"pr_number", type=int, help="the number of the pull request to check"
)
parser.add_argument(
"config_path",
type=Path,
help="the path to the reviewers config file",
)
super().__init__(parser) super().__init__(parser)

53
tools/generate_help.py Executable file
View file

@ -0,0 +1,53 @@
#!/usr/bin/env python3
import os
import re
import subprocess
from typing import Union
def run_mako_command(command: Union[str, list[str]]) -> str:
subprocess_cmd = ["poetry", "run", "ryujinx-mako"]
if isinstance(command, str):
subprocess_cmd.append(command)
elif isinstance(command, list):
subprocess_cmd.extend(command)
else:
raise TypeError(command)
env = os.environ.copy()
env["MAKO_DRY_RUN"] = "1"
process = subprocess.run(
subprocess_cmd, stdout=subprocess.PIPE, check=True, env=env
)
return process.stdout.decode()
def print_help(name: str, output: str, level=3):
headline_prefix = "#" * level
print(f"{headline_prefix} {name}\n")
print("```")
print(output.rstrip())
print("```\n")
general_help = run_mako_command("--help")
for line in general_help.splitlines():
subcommands = re.match(r" {2}\{(.+)}", line)
if subcommands:
break
else:
subcommands = None
if not subcommands:
print("Could not find subcommands in general help output:")
print(general_help)
exit(1)
subcommands = subcommands.group(1).split(",")
print_help("Available commands", general_help, 2)
for subcommand in subcommands:
print_help(subcommand, run_mako_command([subcommand, "--help"]))