Contributing
🤝 Contributing#
We welcome contributions! Whether it's adding new benchmarks, supporting new models, or submitting your hardware results.
Development Setup#
-
Fork and clone the repository
-
Install dependencies including dev tools
-
Install pre-commit hooks
This sets up automatic code quality checks that run before each commit:
- ruff — Fast Python linter and formatter
- mypy — Static type checking
- bandit — Security vulnerability scanner
- Standard checks (trailing whitespace, YAML syntax, etc.)
Making Changes#
-
Create a new branch
-
Make your changes
- Write code following the existing patterns
- Add type hints where applicable
-
Update documentation if needed
-
Test your changes
Available Makefile commands:
make— Generate benchmark results tables (default)make generate— Generate benchmark results tablesmake format— Run pre-commit hooks on all filesmake lint— Run ruff linter onlymake clean— Clean Python cache files-
make help— Show all available commands -
Commit your changes
Pre-commit hooks will automatically: - Format your code - Check for type errors - Scan for security issues - Fix common issues (trailing whitespace, etc.)
If any check fails, fix the issues and commit again.
- Push and create a Pull Request
Code Quality Standards#
All contributions must pass:
- Ruff linting and formatting
- Mypy type checking
- Bandit security checks
These are enforced automatically via pre-commit hooks.
Adding New Benchmarks#
See CLAUDE.md for detailed instructions on: - Adding new models to existing benchmarks - Creating new benchmark categories - Data loading patterns - Memory management best practices
Tip: Add CLAUDE.md when working with your AI coding assistant — it helps provide full project context.