diff --git a/.deepwork/jobs/pr_docs_update/AGENTS.md b/.deepwork/jobs/pr_docs_update/AGENTS.md new file mode 100644 index 00000000..3650e61b --- /dev/null +++ b/.deepwork/jobs/pr_docs_update/AGENTS.md @@ -0,0 +1,70 @@ +# Project Context for pr_docs_update + +This job automatically updates documentation by analyzing recently merged pull requests. + +## Purpose + +The `pr_docs_update` job helps maintain up-to-date AGENTS.md files and copilot-specific instructions by: +- Fetching recently merged PRs from the GitHub repository +- Analyzing PR content for documentation-relevant changes +- Updating appropriate documentation files +- Tracking processed PRs to enable incremental updates + +## Dual Location Maintenance + +**Important**: This is an example job that can be used in any repository. If you're working on the DeepWork repository itself and want to make this a standard job, follow the dual location pattern: + +1. **Source of truth**: `src/deepwork/standard_jobs/pr_docs_update/` + - Make changes here first + - Tracked in version control + +2. **Working copy**: `.deepwork/jobs/pr_docs_update/` + - Updated from source using `deepwork install` + - Used by `deepwork sync` to generate commands + +## File Organization + +``` +pr_docs_update/ +├── AGENTS.md # This file +├── job.yml # Job definition +└── steps/ + ├── fetch_prs.md # Fetch merged PRs from GitHub + ├── analyze_prs.md # Analyze PR content + └── update_docs.md # Update documentation files +``` + +## How It Works + +1. **State Tracking**: Uses `.deepwork/pr_docs_state.json` to track processed PRs +2. **Incremental Processing**: Only processes new PRs, skips already-processed ones +3. **Flexible Configuration**: Can specify PR count and state file location +4. **Multiple Runs**: Can be run repeatedly to go deeper into PR history + +## Usage Notes + +- Run periodically (e.g., weekly) to keep documentation current +- Increase `pr_count` parameter to process more PRs at once +- State file prevents duplicate processing across runs +- Can be used in any repository with GitHub PRs + +## State File Format + +```json +{ + "processed_prs": [123, 456, 789], + "last_update": "2026-01-19T07:00:00Z", + "total_processed": 3 +} +``` + +## Requirements + +- GitHub API access (uses GitHub MCP tools) +- Write access to documentation files +- Git repository with GitHub remote + +## Last Updated + +- Date: 2026-01-19 +- From: Initial job creation diff --git a/.deepwork/jobs/pr_docs_update/USAGE.md b/.deepwork/jobs/pr_docs_update/USAGE.md new file mode 100644 index 00000000..cb6ac3c6 --- /dev/null +++ b/.deepwork/jobs/pr_docs_update/USAGE.md @@ -0,0 +1,162 @@ +# PR Documentation Update Job - Example Usage + +This document provides example usage scenarios for the `pr_docs_update` job. + +## Overview + +The `pr_docs_update` job automatically updates documentation by analyzing recently merged pull requests. It's designed to: +- Keep AGENTS.md files up to date with code changes +- Update copilot-specific instructions when relevant +- Track processed PRs to avoid duplicates +- Enable incremental processing across multiple runs + +## Basic Usage + +### First Run - Process Last 10 PRs + +```bash +# Run all steps to process the 10 most recent merged PRs +/pr_docs_update.fetch_prs +/pr_docs_update.analyze_prs +/pr_docs_update.update_docs +``` + +The job will: +1. Fetch the 10 most recent merged PRs +2. Filter out any already processed (from state file) +3. Analyze each PR for documentation relevance +4. Update appropriate documentation files +5. Save processed PR numbers to state file + +### Subsequent Runs - Process More PRs + +```bash +# Run with a higher count to go deeper into history +# The state file automatically skips already-processed PRs +/pr_docs_update.fetch_prs pr_count=20 +/pr_docs_update.analyze_prs +/pr_docs_update.update_docs +``` + +## Advanced Usage + +### Custom State File Location + +If you want to track state separately for different purposes: + +```bash +/pr_docs_update.fetch_prs state_file=".deepwork/pr_docs_copilot_state.json" +/pr_docs_update.analyze_prs +/pr_docs_update.update_docs state_file=".deepwork/pr_docs_copilot_state.json" +``` + +### Reprocessing Specific PRs + +To reprocess specific PRs, edit the state file: + +```bash +# View current state +cat .deepwork/pr_docs_state.json + +# Remove specific PR numbers you want to reprocess +# Edit the file and remove those numbers from the "processed_prs" array +``` + +### Processing Large Batches + +For initial setup or catching up after a long time: + +```bash +# Process 50 PRs at once +/pr_docs_update.fetch_prs pr_count=50 +/pr_docs_update.analyze_prs +/pr_docs_update.update_docs +``` + +## State File Structure + +The state file (`.deepwork/pr_docs_state.json`) tracks processed PRs: + +```json +{ + "processed_prs": [123, 456, 789], + "last_update": "2026-01-19T07:00:00Z", + "total_processed": 3 +} +``` + +## What Gets Updated? + +### AGENTS.md Files + +For general code changes that all agents should know about: +- New jobs or workflows +- Codebase structure changes +- New conventions or patterns +- Important context for working with the code + +Located at: `.deepwork/jobs/[job_name]/AGENTS.md` + +### Copilot-Specific Instructions + +For changes specific to GitHub Copilot integration: +- Copilot-specific features +- Copilot workflow changes +- Copilot tool usage patterns + +Located at: `.github/copilot/` or similar copilot instruction locations + +## Scheduling Recommendations + +- **Weekly**: Run with default settings (10 PRs) to stay current +- **After Major Releases**: Run with higher count (30-50 PRs) +- **Initial Setup**: Process all recent PRs in batches + +## Integration with Existing Workflows + +This job can be combined with other DeepWork jobs: + +```bash +# Update docs from PRs, then commit the changes +/pr_docs_update.fetch_prs +/pr_docs_update.analyze_prs +/pr_docs_update.update_docs + +# Review and commit +/commit.test +/commit.lint +/commit.commit_and_push +``` + +## Troubleshooting + +### No New PRs Found + +If all fetched PRs have been processed: +- Increase the `pr_count` parameter +- Check the state file to see what's been processed +- Consider clearing the state file to reprocess all PRs + +### GitHub API Rate Limiting + +If you hit rate limits: +- Reduce the `pr_count` parameter +- Wait for the rate limit to reset +- Use authenticated GitHub API access if available + +### Documentation Files Not Found + +If expected documentation files don't exist: +- The job will notify you about missing files +- Create the necessary AGENTS.md or instruction files +- Re-run the update step + +## Example Output + +After running the job, you'll have: + +1. **pr_list.json**: List of fetched PRs (filtered by state) +2. **pr_analysis.md**: Analysis of each PR with recommendations +3. **updated_files_list.md**: Summary of what was updated +4. **Updated documentation files**: AGENTS.md or copilot instructions with new content +5. **Updated state file**: Tracking newly processed PRs diff --git a/.deepwork/jobs/pr_docs_update/job.yml b/.deepwork/jobs/pr_docs_update/job.yml new file mode 100644 index 00000000..d057b4da --- /dev/null +++ b/.deepwork/jobs/pr_docs_update/job.yml @@ -0,0 +1,68 @@ +name: pr_docs_update +version: "1.0.0" +summary: "Update documentation from recently merged PRs" +description: | + A workflow that analyzes recently merged pull requests and updates either AGENTS.md + or copilot-specific instructions based on the PR content. Tracks processed PRs to + avoid duplicates and can be run multiple times to go further back in PR history. + + This job helps maintain up-to-date documentation by: + 1. Fetching recently merged PRs from GitHub + 2. Analyzing PR content to identify documentation-relevant changes + 3. Updating AGENTS.md (for general context) or copilot instructions (for copilot-specific changes) + 4. Tracking which PRs have been processed to enable incremental updates + + Use this job periodically to keep documentation in sync with code changes. + +changelog: + - version: "1.0.0" + changes: "Initial job creation" + +steps: + - id: fetch_prs + name: "Fetch Recently Merged PRs" + description: "Retrieve list of recently merged pull requests from GitHub" + instructions_file: steps/fetch_prs.md + inputs: + - name: pr_count + description: "Number of recent PRs to fetch (default: 10)" + - name: state_file + description: "Path to state file tracking processed PRs (default: .deepwork/pr_docs_state.json)" + outputs: + - pr_list.json + dependencies: [] + + - id: analyze_prs + name: "Analyze PR Content" + description: "Analyze PR content to identify documentation-relevant changes" + instructions_file: steps/analyze_prs.md + inputs: + - file: pr_list.json + from_step: fetch_prs + outputs: + - pr_analysis.md + dependencies: + - fetch_prs + + - id: update_docs + name: "Update Documentation" + description: "Update AGENTS.md or copilot instructions based on PR analysis" + instructions_file: steps/update_docs.md + inputs: + - file: pr_analysis.md + from_step: analyze_prs + - name: state_file + description: "Path to state file tracking processed PRs (default: .deepwork/pr_docs_state.json)" + outputs: + - updated_files_list.md + dependencies: + - analyze_prs + hooks: + after_agent: + - prompt: | + Verify the documentation update is complete: + 1. Relevant documentation files (AGENTS.md or copilot instructions) were identified + 2. Documentation was updated with insights from analyzed PRs + 3. State file was updated with processed PR numbers + 4. Changes are ready to be committed + If ALL criteria are met, include `✓ Quality Criteria Met`. diff --git a/.deepwork/jobs/pr_docs_update/steps/analyze_prs.md b/.deepwork/jobs/pr_docs_update/steps/analyze_prs.md new file mode 100644 index 00000000..84ac31d0 --- /dev/null +++ b/.deepwork/jobs/pr_docs_update/steps/analyze_prs.md @@ -0,0 +1,137 @@ +# Analyze PR Content + +## Objective + +Analyze the content of unprocessed pull requests to identify which ones contain changes relevant to documentation updates, specifically for AGENTS.md files or copilot-specific instructions. + +## Task + +Review each PR from the input list and determine what documentation updates are needed based on the PR's changes, title, description, and files modified. + +### Step 1: Load PR List + +1. **Read the input file** `pr_list.json` + - Parse the JSON structure + - Extract the list of PRs to analyze + +2. **Verify the data** + - Ensure PRs have all required fields + - Note the repository context + +### Step 2: Analyze Each PR + +For each PR in the list, evaluate its relevance to documentation: + +1. **Examine Changed Files** + - Look for patterns that suggest documentation impact: + - New features or capabilities added + - Changes to job definitions or workflows + - API or interface modifications + - New standard jobs or tools + - Configuration changes + - Agent or LLM interaction changes + +2. **Review PR Title and Description** + - Identify the purpose of the PR + - Note any documentation mentions + - Look for keywords like: "add", "new", "feature", "job", "workflow", "agent", "copilot" + +3. **Categorize the PR** + - **General documentation**: Changes that affect all agents (→ AGENTS.md) + - **Copilot-specific**: Changes specific to GitHub Copilot integration (→ copilot instructions) + - **Not relevant**: Changes with no documentation impact + - **Unclear**: Needs further investigation + +4. **Extract Documentation Insights** + For relevant PRs, identify: + - What was added or changed? + - What should agents know about this change? + - Which documentation file should be updated? + - What specific information should be added? + +### Step 3: Determine Documentation Updates + +For each relevant PR, create a documentation recommendation: + +1. **For AGENTS.md updates**: + - New job or workflow information + - Codebase structure changes + - New conventions or patterns + - Important context for all agents + +2. **For copilot-specific instructions**: + - Copilot-specific features or integrations + - Copilot workflow changes + - Copilot tool usage patterns + +3. **Documentation entry format**: + - Clear, concise description + - Reference to the PR number + - Suggested location in the documentation + - Specific text to add or update + +### Step 4: Create Analysis Report + +Create `pr_analysis.md` with the following structure: + +```markdown +# PR Documentation Analysis + +**Repository**: owner/repo-name +**Analysis Date**: 2026-01-19 +**PRs Analyzed**: 5 +**Documentation Updates Needed**: 3 + +## PRs Requiring Documentation Updates + +### PR #123: Add new feature +- **Merge Date**: 2026-01-15 +- **Author**: username +- **URL**: https://github.com/owner/repo/pull/123 +- **Category**: General Documentation (AGENTS.md) +- **Summary**: This PR adds a new job type for automated testing +- **Documentation Update**: + - File: `.deepwork/jobs/[job_name]/AGENTS.md` + - Section: Job Structure + - Content: Document the new automated testing job available in the repository + - Rationale: Agents should be aware of this capability for future task planning + +### PR #456: Update copilot integration +- **Merge Date**: 2026-01-14 +- **Author**: username2 +- **URL**: https://github.com/owner/repo/pull/456 +- **Category**: Copilot-Specific Instructions +- **Summary**: Enhanced copilot workflow with new commands +- **Documentation Update**: + - File: Copilot-specific instruction files + - Section: Available Commands + - Content: Document the new `/copilot-command` syntax and usage + - Rationale: Copilot agents need to know about new command capabilities + +## PRs Not Requiring Updates + +### PR #789: Fix typo in README +- **Reason**: Minor documentation fix, no agent-relevant changes +``` + +## Output Format + +- **File**: `pr_analysis.md` +- **Content**: Markdown document with analysis results and recommendations +- **Location**: Current working directory + +## Quality Criteria + +- ✓ All PRs from input list analyzed +- ✓ Each PR categorized (general docs, copilot-specific, not relevant, or unclear) +- ✓ Documentation updates clearly specified with target file and content +- ✓ Rationale provided for each recommended update +- ✓ Summary statistics included (PRs analyzed, updates needed) +- ✓ Clear, actionable recommendations for the next step + +## Notes + +- Focus on changes that provide value to AI agents working with the codebase +- Avoid documenting trivial changes (typos, formatting, etc.) +- When in doubt, err on the side of including information that might be helpful +- Consider the perspective of an agent encountering this codebase for the first time diff --git a/.deepwork/jobs/pr_docs_update/steps/fetch_prs.md b/.deepwork/jobs/pr_docs_update/steps/fetch_prs.md new file mode 100644 index 00000000..1c53ee8e --- /dev/null +++ b/.deepwork/jobs/pr_docs_update/steps/fetch_prs.md @@ -0,0 +1,108 @@ +# Fetch Recently Merged PRs + +## Objective + +Retrieve a list of recently merged pull requests from GitHub and filter out any that have already been processed, based on a state file. + +## Task + +Fetch recently merged PRs from the current repository and prepare a list of unprocessed PRs for analysis. + +### Step 1: Understand the Parameters + +1. **PR Count**: Determine how many recent PRs to fetch + - Default: 10 PRs + - If user specified a count, use that value + - Can fetch more to go deeper into history + +2. **State File**: Load the state file to track processed PRs + - Default location: `.deepwork/pr_docs_state.json` + - If file doesn't exist, create it with an empty list + - File format: `{"processed_prs": [123, 456, 789]}` + +### Step 2: Fetch Merged PRs from GitHub + +Use GitHub tools available to you to fetch recently merged pull requests: + +1. **Get the repository information** + - Repository owner and name from current git repository + - Use `git remote -v` to identify the GitHub repository + +2. **Fetch merged PRs** + - Use GitHub API or available GitHub tools to list merged pull requests + - Sort by merge date (most recent first) + - Fetch the number requested (default 10, or user-specified count) + +3. **For each PR, collect**: + - PR number + - Title + - Description/body + - Merge date + - Author + - Labels (if any) + - Changed files (summary) + - Link to the PR + +### Step 3: Filter Out Already Processed PRs + +1. **Load the state file** + - Read `.deepwork/pr_docs_state.json` (or user-specified path) + - Parse the JSON to get the list of processed PR numbers + - If file doesn't exist, treat it as empty (no processed PRs) + +2. **Filter the PR list** + - Remove PRs whose numbers are in the `processed_prs` list + - Keep only unprocessed PRs + +3. **Handle the case of no new PRs** + - If all fetched PRs have been processed, inform the user + - Suggest fetching more PRs by increasing the count + - Recommend using a larger `pr_count` value to go deeper into history + +### Step 4: Create Output File + +Create `pr_list.json` with the following structure: + +```json +{ + "repository": "owner/repo-name", + "fetch_date": "2026-01-19T07:00:00Z", + "total_fetched": 10, + "new_prs": 5, + "prs": [ + { + "number": 123, + "title": "Add new feature", + "description": "This PR adds...", + "merge_date": "2026-01-15T10:30:00Z", + "author": "username", + "labels": ["enhancement", "documentation"], + "changed_files": ["src/feature.py", "docs/README.md"], + "url": "https://github.com/owner/repo/pull/123" + } + ] +} +``` + +## Output Format + +- **File**: `pr_list.json` +- **Content**: JSON structure with repository info and list of unprocessed merged PRs +- **Location**: Current working directory + +## Quality Criteria + +- ✓ Repository information correctly identified from git remote +- ✓ At least the requested number of PRs fetched (unless fewer exist) +- ✓ State file loaded (or created if missing) +- ✓ Already-processed PRs filtered out +- ✓ PR data includes all required fields (number, title, description, merge_date, author, url) +- ✓ Output file created with valid JSON structure +- ✓ If no new PRs found, user informed about options to fetch more + +## Notes + +- This step uses GitHub API access, which may require authentication +- If you encounter rate limiting, inform the user +- The state file prevents duplicate processing across multiple runs +- Users can delete or edit the state file to reprocess specific PRs if needed diff --git a/.deepwork/jobs/pr_docs_update/steps/update_docs.md b/.deepwork/jobs/pr_docs_update/steps/update_docs.md new file mode 100644 index 00000000..f3ba3a1e --- /dev/null +++ b/.deepwork/jobs/pr_docs_update/steps/update_docs.md @@ -0,0 +1,168 @@ +# Update Documentation + +## Objective + +Update AGENTS.md files or copilot-specific instructions based on the PR analysis, and track processed PRs in the state file to prevent duplicate processing in future runs. + +## Task + +Apply the documentation recommendations from the analysis step, update the appropriate files, and maintain the state file for tracking. + +### Step 1: Load the Analysis + +1. **Read `pr_analysis.md`** + - Parse the recommendations + - Identify which files need updates + - Group updates by target file + +2. **Load the state file** + - Read `.deepwork/pr_docs_state.json` (or user-specified path) + - Prepare to add newly processed PR numbers + +### Step 2: Identify Target Documentation Files + +Based on the analysis, determine which documentation files to update: + +1. **AGENTS.md files** + - Check for existing AGENTS.md files in relevant job directories + - Typical locations: + - `.deepwork/jobs/[job_name]/AGENTS.md` + - `src/deepwork/standard_jobs/[job_name]/AGENTS.md` + - If no AGENTS.md exists for relevant jobs, consider creating one + +2. **Copilot-specific instructions** + - Look for copilot instruction files + - Common locations might include: + - `.github/copilot/` + - `.copilot/` + - Other copilot-specific configuration locations + - If no copilot instructions exist and updates are needed, note this for the user + +3. **Create an update plan** + - List all files to be updated + - Organize updates by file to minimize context switching + +### Step 3: Update Documentation Files + +For each documentation file identified: + +1. **Review existing content** + - Read the current file content + - Understand the existing structure and sections + - Identify where new content should be added + +2. **Apply updates** + - Add new information based on PR analysis + - Include PR reference numbers for traceability + - Follow the existing format and style + - Maintain chronological order if applicable + - Keep entries clear and concise + +3. **AGENTS.md format example**: + ```markdown + ## Last Updated + + - Date: 2026-01-19 + - From: PR documentation sync (PRs #123, #456) + + ## Recent Changes + + ### PR #123: New Automated Testing Job + - A new job type for automated testing is now available + - Location: `.deepwork/jobs/automated_test/` + - Use this for test automation workflows + - See job.yml for full specification + ``` + +4. **Handle multiple file updates** + - Update all relevant files identified in the analysis + - Maintain consistency across updates + - Ensure cross-references are accurate + +### Step 4: Update State File + +1. **Collect processed PR numbers** + - Extract PR numbers from the analysis + - Include all PRs (both relevant and not relevant) + - This ensures we don't reprocess any PRs + +2. **Update the state file** + - Load existing `processed_prs` list + - Add new PR numbers to the list + - Remove duplicates if any + - Sort the list (optional, for readability) + +3. **Write the updated state file** + ```json + { + "processed_prs": [123, 456, 789, 1011, 1213], + "last_update": "2026-01-19T07:00:00Z", + "total_processed": 5 + } + ``` + +### Step 5: Create Summary Report + +Create `updated_files_list.md` documenting what was done: + +```markdown +# Documentation Update Summary + +**Date**: 2026-01-19 +**PRs Processed**: 5 (PRs #123, #456, #789, #1011, #1213) +**Files Updated**: 2 + +## Updated Files + +### .deepwork/jobs/deepwork_jobs/AGENTS.md +- Added information about new automated testing job (PR #123) +- Updated last modified date + +### .github/copilot/instructions.md +- Documented new copilot command syntax (PR #456) +- Added usage examples + +## State File Updated + +- State file: `.deepwork/pr_docs_state.json` +- New processed PRs added: 5 +- Total PRs tracked: 15 + +## PRs Processed But Not Documented + +- PR #789: Minor typo fix (not relevant) +- PR #1011: Internal refactoring (no documentation impact) +- PR #1213: CI configuration (no agent-visible changes) + +## Next Steps + +To process more PRs from further back in history, run this job again with: +- Increase `pr_count` parameter (e.g., 20 or 30) +- The state file will automatically skip already processed PRs +``` + +## Output Format + +- **File**: `updated_files_list.md` +- **Content**: Summary of changes made and files updated +- **Location**: Current working directory + +## Quality Criteria + +- ✓ All relevant documentation files identified and updated +- ✓ Updates include PR references for traceability +- ✓ Existing file structure and format maintained +- ✓ State file updated with all processed PR numbers +- ✓ State file includes metadata (last_update, total_processed) +- ✓ Summary report created with clear list of changes +- ✓ Changes are ready to be committed +- ✓ If no files exist for updates, user is informed + +## Notes + +- If AGENTS.md files don't exist in expected locations, ask the user if they should be created +- Maintain the existing tone and style of documentation files +- Include enough context so future readers understand the changes +- The state file enables incremental processing across multiple runs +- Users can manually edit the state file to reprocess specific PRs if needed +- Consider running a git diff to show the user what changed before committing diff --git a/src/deepwork/standard_jobs/pr_docs_update/AGENTS.md b/src/deepwork/standard_jobs/pr_docs_update/AGENTS.md new file mode 100644 index 00000000..3650e61b --- /dev/null +++ b/src/deepwork/standard_jobs/pr_docs_update/AGENTS.md @@ -0,0 +1,70 @@ +# Project Context for pr_docs_update + +This job automatically updates documentation by analyzing recently merged pull requests. + +## Purpose + +The `pr_docs_update` job helps maintain up-to-date AGENTS.md files and copilot-specific instructions by: +- Fetching recently merged PRs from the GitHub repository +- Analyzing PR content for documentation-relevant changes +- Updating appropriate documentation files +- Tracking processed PRs to enable incremental updates + +## Dual Location Maintenance + +**Important**: This is an example job that can be used in any repository. If you're working on the DeepWork repository itself and want to make this a standard job, follow the dual location pattern: + +1. **Source of truth**: `src/deepwork/standard_jobs/pr_docs_update/` + - Make changes here first + - Tracked in version control + +2. **Working copy**: `.deepwork/jobs/pr_docs_update/` + - Updated from source using `deepwork install` + - Used by `deepwork sync` to generate commands + +## File Organization + +``` +pr_docs_update/ +├── AGENTS.md # This file +├── job.yml # Job definition +└── steps/ + ├── fetch_prs.md # Fetch merged PRs from GitHub + ├── analyze_prs.md # Analyze PR content + └── update_docs.md # Update documentation files +``` + +## How It Works + +1. **State Tracking**: Uses `.deepwork/pr_docs_state.json` to track processed PRs +2. **Incremental Processing**: Only processes new PRs, skips already-processed ones +3. **Flexible Configuration**: Can specify PR count and state file location +4. **Multiple Runs**: Can be run repeatedly to go deeper into PR history + +## Usage Notes + +- Run periodically (e.g., weekly) to keep documentation current +- Increase `pr_count` parameter to process more PRs at once +- State file prevents duplicate processing across runs +- Can be used in any repository with GitHub PRs + +## State File Format + +```json +{ + "processed_prs": [123, 456, 789], + "last_update": "2026-01-19T07:00:00Z", + "total_processed": 3 +} +``` + +## Requirements + +- GitHub API access (uses GitHub MCP tools) +- Write access to documentation files +- Git repository with GitHub remote + +## Last Updated + +- Date: 2026-01-19 +- From: Initial job creation diff --git a/src/deepwork/standard_jobs/pr_docs_update/USAGE.md b/src/deepwork/standard_jobs/pr_docs_update/USAGE.md new file mode 100644 index 00000000..cb6ac3c6 --- /dev/null +++ b/src/deepwork/standard_jobs/pr_docs_update/USAGE.md @@ -0,0 +1,162 @@ +# PR Documentation Update Job - Example Usage + +This document provides example usage scenarios for the `pr_docs_update` job. + +## Overview + +The `pr_docs_update` job automatically updates documentation by analyzing recently merged pull requests. It's designed to: +- Keep AGENTS.md files up to date with code changes +- Update copilot-specific instructions when relevant +- Track processed PRs to avoid duplicates +- Enable incremental processing across multiple runs + +## Basic Usage + +### First Run - Process Last 10 PRs + +```bash +# Run all steps to process the 10 most recent merged PRs +/pr_docs_update.fetch_prs +/pr_docs_update.analyze_prs +/pr_docs_update.update_docs +``` + +The job will: +1. Fetch the 10 most recent merged PRs +2. Filter out any already processed (from state file) +3. Analyze each PR for documentation relevance +4. Update appropriate documentation files +5. Save processed PR numbers to state file + +### Subsequent Runs - Process More PRs + +```bash +# Run with a higher count to go deeper into history +# The state file automatically skips already-processed PRs +/pr_docs_update.fetch_prs pr_count=20 +/pr_docs_update.analyze_prs +/pr_docs_update.update_docs +``` + +## Advanced Usage + +### Custom State File Location + +If you want to track state separately for different purposes: + +```bash +/pr_docs_update.fetch_prs state_file=".deepwork/pr_docs_copilot_state.json" +/pr_docs_update.analyze_prs +/pr_docs_update.update_docs state_file=".deepwork/pr_docs_copilot_state.json" +``` + +### Reprocessing Specific PRs + +To reprocess specific PRs, edit the state file: + +```bash +# View current state +cat .deepwork/pr_docs_state.json + +# Remove specific PR numbers you want to reprocess +# Edit the file and remove those numbers from the "processed_prs" array +``` + +### Processing Large Batches + +For initial setup or catching up after a long time: + +```bash +# Process 50 PRs at once +/pr_docs_update.fetch_prs pr_count=50 +/pr_docs_update.analyze_prs +/pr_docs_update.update_docs +``` + +## State File Structure + +The state file (`.deepwork/pr_docs_state.json`) tracks processed PRs: + +```json +{ + "processed_prs": [123, 456, 789], + "last_update": "2026-01-19T07:00:00Z", + "total_processed": 3 +} +``` + +## What Gets Updated? + +### AGENTS.md Files + +For general code changes that all agents should know about: +- New jobs or workflows +- Codebase structure changes +- New conventions or patterns +- Important context for working with the code + +Located at: `.deepwork/jobs/[job_name]/AGENTS.md` + +### Copilot-Specific Instructions + +For changes specific to GitHub Copilot integration: +- Copilot-specific features +- Copilot workflow changes +- Copilot tool usage patterns + +Located at: `.github/copilot/` or similar copilot instruction locations + +## Scheduling Recommendations + +- **Weekly**: Run with default settings (10 PRs) to stay current +- **After Major Releases**: Run with higher count (30-50 PRs) +- **Initial Setup**: Process all recent PRs in batches + +## Integration with Existing Workflows + +This job can be combined with other DeepWork jobs: + +```bash +# Update docs from PRs, then commit the changes +/pr_docs_update.fetch_prs +/pr_docs_update.analyze_prs +/pr_docs_update.update_docs + +# Review and commit +/commit.test +/commit.lint +/commit.commit_and_push +``` + +## Troubleshooting + +### No New PRs Found + +If all fetched PRs have been processed: +- Increase the `pr_count` parameter +- Check the state file to see what's been processed +- Consider clearing the state file to reprocess all PRs + +### GitHub API Rate Limiting + +If you hit rate limits: +- Reduce the `pr_count` parameter +- Wait for the rate limit to reset +- Use authenticated GitHub API access if available + +### Documentation Files Not Found + +If expected documentation files don't exist: +- The job will notify you about missing files +- Create the necessary AGENTS.md or instruction files +- Re-run the update step + +## Example Output + +After running the job, you'll have: + +1. **pr_list.json**: List of fetched PRs (filtered by state) +2. **pr_analysis.md**: Analysis of each PR with recommendations +3. **updated_files_list.md**: Summary of what was updated +4. **Updated documentation files**: AGENTS.md or copilot instructions with new content +5. **Updated state file**: Tracking newly processed PRs diff --git a/src/deepwork/standard_jobs/pr_docs_update/job.yml b/src/deepwork/standard_jobs/pr_docs_update/job.yml new file mode 100644 index 00000000..d057b4da --- /dev/null +++ b/src/deepwork/standard_jobs/pr_docs_update/job.yml @@ -0,0 +1,68 @@ +name: pr_docs_update +version: "1.0.0" +summary: "Update documentation from recently merged PRs" +description: | + A workflow that analyzes recently merged pull requests and updates either AGENTS.md + or copilot-specific instructions based on the PR content. Tracks processed PRs to + avoid duplicates and can be run multiple times to go further back in PR history. + + This job helps maintain up-to-date documentation by: + 1. Fetching recently merged PRs from GitHub + 2. Analyzing PR content to identify documentation-relevant changes + 3. Updating AGENTS.md (for general context) or copilot instructions (for copilot-specific changes) + 4. Tracking which PRs have been processed to enable incremental updates + + Use this job periodically to keep documentation in sync with code changes. + +changelog: + - version: "1.0.0" + changes: "Initial job creation" + +steps: + - id: fetch_prs + name: "Fetch Recently Merged PRs" + description: "Retrieve list of recently merged pull requests from GitHub" + instructions_file: steps/fetch_prs.md + inputs: + - name: pr_count + description: "Number of recent PRs to fetch (default: 10)" + - name: state_file + description: "Path to state file tracking processed PRs (default: .deepwork/pr_docs_state.json)" + outputs: + - pr_list.json + dependencies: [] + + - id: analyze_prs + name: "Analyze PR Content" + description: "Analyze PR content to identify documentation-relevant changes" + instructions_file: steps/analyze_prs.md + inputs: + - file: pr_list.json + from_step: fetch_prs + outputs: + - pr_analysis.md + dependencies: + - fetch_prs + + - id: update_docs + name: "Update Documentation" + description: "Update AGENTS.md or copilot instructions based on PR analysis" + instructions_file: steps/update_docs.md + inputs: + - file: pr_analysis.md + from_step: analyze_prs + - name: state_file + description: "Path to state file tracking processed PRs (default: .deepwork/pr_docs_state.json)" + outputs: + - updated_files_list.md + dependencies: + - analyze_prs + hooks: + after_agent: + - prompt: | + Verify the documentation update is complete: + 1. Relevant documentation files (AGENTS.md or copilot instructions) were identified + 2. Documentation was updated with insights from analyzed PRs + 3. State file was updated with processed PR numbers + 4. Changes are ready to be committed + If ALL criteria are met, include `✓ Quality Criteria Met`. diff --git a/src/deepwork/standard_jobs/pr_docs_update/steps/analyze_prs.md b/src/deepwork/standard_jobs/pr_docs_update/steps/analyze_prs.md new file mode 100644 index 00000000..84ac31d0 --- /dev/null +++ b/src/deepwork/standard_jobs/pr_docs_update/steps/analyze_prs.md @@ -0,0 +1,137 @@ +# Analyze PR Content + +## Objective + +Analyze the content of unprocessed pull requests to identify which ones contain changes relevant to documentation updates, specifically for AGENTS.md files or copilot-specific instructions. + +## Task + +Review each PR from the input list and determine what documentation updates are needed based on the PR's changes, title, description, and files modified. + +### Step 1: Load PR List + +1. **Read the input file** `pr_list.json` + - Parse the JSON structure + - Extract the list of PRs to analyze + +2. **Verify the data** + - Ensure PRs have all required fields + - Note the repository context + +### Step 2: Analyze Each PR + +For each PR in the list, evaluate its relevance to documentation: + +1. **Examine Changed Files** + - Look for patterns that suggest documentation impact: + - New features or capabilities added + - Changes to job definitions or workflows + - API or interface modifications + - New standard jobs or tools + - Configuration changes + - Agent or LLM interaction changes + +2. **Review PR Title and Description** + - Identify the purpose of the PR + - Note any documentation mentions + - Look for keywords like: "add", "new", "feature", "job", "workflow", "agent", "copilot" + +3. **Categorize the PR** + - **General documentation**: Changes that affect all agents (→ AGENTS.md) + - **Copilot-specific**: Changes specific to GitHub Copilot integration (→ copilot instructions) + - **Not relevant**: Changes with no documentation impact + - **Unclear**: Needs further investigation + +4. **Extract Documentation Insights** + For relevant PRs, identify: + - What was added or changed? + - What should agents know about this change? + - Which documentation file should be updated? + - What specific information should be added? + +### Step 3: Determine Documentation Updates + +For each relevant PR, create a documentation recommendation: + +1. **For AGENTS.md updates**: + - New job or workflow information + - Codebase structure changes + - New conventions or patterns + - Important context for all agents + +2. **For copilot-specific instructions**: + - Copilot-specific features or integrations + - Copilot workflow changes + - Copilot tool usage patterns + +3. **Documentation entry format**: + - Clear, concise description + - Reference to the PR number + - Suggested location in the documentation + - Specific text to add or update + +### Step 4: Create Analysis Report + +Create `pr_analysis.md` with the following structure: + +```markdown +# PR Documentation Analysis + +**Repository**: owner/repo-name +**Analysis Date**: 2026-01-19 +**PRs Analyzed**: 5 +**Documentation Updates Needed**: 3 + +## PRs Requiring Documentation Updates + +### PR #123: Add new feature +- **Merge Date**: 2026-01-15 +- **Author**: username +- **URL**: https://github.com/owner/repo/pull/123 +- **Category**: General Documentation (AGENTS.md) +- **Summary**: This PR adds a new job type for automated testing +- **Documentation Update**: + - File: `.deepwork/jobs/[job_name]/AGENTS.md` + - Section: Job Structure + - Content: Document the new automated testing job available in the repository + - Rationale: Agents should be aware of this capability for future task planning + +### PR #456: Update copilot integration +- **Merge Date**: 2026-01-14 +- **Author**: username2 +- **URL**: https://github.com/owner/repo/pull/456 +- **Category**: Copilot-Specific Instructions +- **Summary**: Enhanced copilot workflow with new commands +- **Documentation Update**: + - File: Copilot-specific instruction files + - Section: Available Commands + - Content: Document the new `/copilot-command` syntax and usage + - Rationale: Copilot agents need to know about new command capabilities + +## PRs Not Requiring Updates + +### PR #789: Fix typo in README +- **Reason**: Minor documentation fix, no agent-relevant changes +``` + +## Output Format + +- **File**: `pr_analysis.md` +- **Content**: Markdown document with analysis results and recommendations +- **Location**: Current working directory + +## Quality Criteria + +- ✓ All PRs from input list analyzed +- ✓ Each PR categorized (general docs, copilot-specific, not relevant, or unclear) +- ✓ Documentation updates clearly specified with target file and content +- ✓ Rationale provided for each recommended update +- ✓ Summary statistics included (PRs analyzed, updates needed) +- ✓ Clear, actionable recommendations for the next step + +## Notes + +- Focus on changes that provide value to AI agents working with the codebase +- Avoid documenting trivial changes (typos, formatting, etc.) +- When in doubt, err on the side of including information that might be helpful +- Consider the perspective of an agent encountering this codebase for the first time diff --git a/src/deepwork/standard_jobs/pr_docs_update/steps/fetch_prs.md b/src/deepwork/standard_jobs/pr_docs_update/steps/fetch_prs.md new file mode 100644 index 00000000..1c53ee8e --- /dev/null +++ b/src/deepwork/standard_jobs/pr_docs_update/steps/fetch_prs.md @@ -0,0 +1,108 @@ +# Fetch Recently Merged PRs + +## Objective + +Retrieve a list of recently merged pull requests from GitHub and filter out any that have already been processed, based on a state file. + +## Task + +Fetch recently merged PRs from the current repository and prepare a list of unprocessed PRs for analysis. + +### Step 1: Understand the Parameters + +1. **PR Count**: Determine how many recent PRs to fetch + - Default: 10 PRs + - If user specified a count, use that value + - Can fetch more to go deeper into history + +2. **State File**: Load the state file to track processed PRs + - Default location: `.deepwork/pr_docs_state.json` + - If file doesn't exist, create it with an empty list + - File format: `{"processed_prs": [123, 456, 789]}` + +### Step 2: Fetch Merged PRs from GitHub + +Use GitHub tools available to you to fetch recently merged pull requests: + +1. **Get the repository information** + - Repository owner and name from current git repository + - Use `git remote -v` to identify the GitHub repository + +2. **Fetch merged PRs** + - Use GitHub API or available GitHub tools to list merged pull requests + - Sort by merge date (most recent first) + - Fetch the number requested (default 10, or user-specified count) + +3. **For each PR, collect**: + - PR number + - Title + - Description/body + - Merge date + - Author + - Labels (if any) + - Changed files (summary) + - Link to the PR + +### Step 3: Filter Out Already Processed PRs + +1. **Load the state file** + - Read `.deepwork/pr_docs_state.json` (or user-specified path) + - Parse the JSON to get the list of processed PR numbers + - If file doesn't exist, treat it as empty (no processed PRs) + +2. **Filter the PR list** + - Remove PRs whose numbers are in the `processed_prs` list + - Keep only unprocessed PRs + +3. **Handle the case of no new PRs** + - If all fetched PRs have been processed, inform the user + - Suggest fetching more PRs by increasing the count + - Recommend using a larger `pr_count` value to go deeper into history + +### Step 4: Create Output File + +Create `pr_list.json` with the following structure: + +```json +{ + "repository": "owner/repo-name", + "fetch_date": "2026-01-19T07:00:00Z", + "total_fetched": 10, + "new_prs": 5, + "prs": [ + { + "number": 123, + "title": "Add new feature", + "description": "This PR adds...", + "merge_date": "2026-01-15T10:30:00Z", + "author": "username", + "labels": ["enhancement", "documentation"], + "changed_files": ["src/feature.py", "docs/README.md"], + "url": "https://github.com/owner/repo/pull/123" + } + ] +} +``` + +## Output Format + +- **File**: `pr_list.json` +- **Content**: JSON structure with repository info and list of unprocessed merged PRs +- **Location**: Current working directory + +## Quality Criteria + +- ✓ Repository information correctly identified from git remote +- ✓ At least the requested number of PRs fetched (unless fewer exist) +- ✓ State file loaded (or created if missing) +- ✓ Already-processed PRs filtered out +- ✓ PR data includes all required fields (number, title, description, merge_date, author, url) +- ✓ Output file created with valid JSON structure +- ✓ If no new PRs found, user informed about options to fetch more + +## Notes + +- This step uses GitHub API access, which may require authentication +- If you encounter rate limiting, inform the user +- The state file prevents duplicate processing across multiple runs +- Users can delete or edit the state file to reprocess specific PRs if needed diff --git a/src/deepwork/standard_jobs/pr_docs_update/steps/update_docs.md b/src/deepwork/standard_jobs/pr_docs_update/steps/update_docs.md new file mode 100644 index 00000000..f3ba3a1e --- /dev/null +++ b/src/deepwork/standard_jobs/pr_docs_update/steps/update_docs.md @@ -0,0 +1,168 @@ +# Update Documentation + +## Objective + +Update AGENTS.md files or copilot-specific instructions based on the PR analysis, and track processed PRs in the state file to prevent duplicate processing in future runs. + +## Task + +Apply the documentation recommendations from the analysis step, update the appropriate files, and maintain the state file for tracking. + +### Step 1: Load the Analysis + +1. **Read `pr_analysis.md`** + - Parse the recommendations + - Identify which files need updates + - Group updates by target file + +2. **Load the state file** + - Read `.deepwork/pr_docs_state.json` (or user-specified path) + - Prepare to add newly processed PR numbers + +### Step 2: Identify Target Documentation Files + +Based on the analysis, determine which documentation files to update: + +1. **AGENTS.md files** + - Check for existing AGENTS.md files in relevant job directories + - Typical locations: + - `.deepwork/jobs/[job_name]/AGENTS.md` + - `src/deepwork/standard_jobs/[job_name]/AGENTS.md` + - If no AGENTS.md exists for relevant jobs, consider creating one + +2. **Copilot-specific instructions** + - Look for copilot instruction files + - Common locations might include: + - `.github/copilot/` + - `.copilot/` + - Other copilot-specific configuration locations + - If no copilot instructions exist and updates are needed, note this for the user + +3. **Create an update plan** + - List all files to be updated + - Organize updates by file to minimize context switching + +### Step 3: Update Documentation Files + +For each documentation file identified: + +1. **Review existing content** + - Read the current file content + - Understand the existing structure and sections + - Identify where new content should be added + +2. **Apply updates** + - Add new information based on PR analysis + - Include PR reference numbers for traceability + - Follow the existing format and style + - Maintain chronological order if applicable + - Keep entries clear and concise + +3. **AGENTS.md format example**: + ```markdown + ## Last Updated + + - Date: 2026-01-19 + - From: PR documentation sync (PRs #123, #456) + + ## Recent Changes + + ### PR #123: New Automated Testing Job + - A new job type for automated testing is now available + - Location: `.deepwork/jobs/automated_test/` + - Use this for test automation workflows + - See job.yml for full specification + ``` + +4. **Handle multiple file updates** + - Update all relevant files identified in the analysis + - Maintain consistency across updates + - Ensure cross-references are accurate + +### Step 4: Update State File + +1. **Collect processed PR numbers** + - Extract PR numbers from the analysis + - Include all PRs (both relevant and not relevant) + - This ensures we don't reprocess any PRs + +2. **Update the state file** + - Load existing `processed_prs` list + - Add new PR numbers to the list + - Remove duplicates if any + - Sort the list (optional, for readability) + +3. **Write the updated state file** + ```json + { + "processed_prs": [123, 456, 789, 1011, 1213], + "last_update": "2026-01-19T07:00:00Z", + "total_processed": 5 + } + ``` + +### Step 5: Create Summary Report + +Create `updated_files_list.md` documenting what was done: + +```markdown +# Documentation Update Summary + +**Date**: 2026-01-19 +**PRs Processed**: 5 (PRs #123, #456, #789, #1011, #1213) +**Files Updated**: 2 + +## Updated Files + +### .deepwork/jobs/deepwork_jobs/AGENTS.md +- Added information about new automated testing job (PR #123) +- Updated last modified date + +### .github/copilot/instructions.md +- Documented new copilot command syntax (PR #456) +- Added usage examples + +## State File Updated + +- State file: `.deepwork/pr_docs_state.json` +- New processed PRs added: 5 +- Total PRs tracked: 15 + +## PRs Processed But Not Documented + +- PR #789: Minor typo fix (not relevant) +- PR #1011: Internal refactoring (no documentation impact) +- PR #1213: CI configuration (no agent-visible changes) + +## Next Steps + +To process more PRs from further back in history, run this job again with: +- Increase `pr_count` parameter (e.g., 20 or 30) +- The state file will automatically skip already processed PRs +``` + +## Output Format + +- **File**: `updated_files_list.md` +- **Content**: Summary of changes made and files updated +- **Location**: Current working directory + +## Quality Criteria + +- ✓ All relevant documentation files identified and updated +- ✓ Updates include PR references for traceability +- ✓ Existing file structure and format maintained +- ✓ State file updated with all processed PR numbers +- ✓ State file includes metadata (last_update, total_processed) +- ✓ Summary report created with clear list of changes +- ✓ Changes are ready to be committed +- ✓ If no files exist for updates, user is informed + +## Notes + +- If AGENTS.md files don't exist in expected locations, ask the user if they should be created +- Maintain the existing tone and style of documentation files +- Include enough context so future readers understand the changes +- The state file enables incremental processing across multiple runs +- Users can manually edit the state file to reprocess specific PRs if needed +- Consider running a git diff to show the user what changed before committing