GPTCode Capabilities & Limitations

Last Updated: December 2025
Current Version: 0.x (MVP)
Overall Autonomy: 64/64 scenarios

This document describes what GPTCode can and cannot do autonomously. Updated with each major release.

What GPTCode Can Do

✅ GitHub Issue Resolution (100% MVAA)

GPTCode can autonomously resolve simple GitHub issues end-to-end:

Languages supported: Go, TypeScript, Python, Elixir, Ruby

Commands:

gptcode issue fix 123       # Fetch and implement
gptcode issue commit 123    # Validate and commit  
gptcode issue push 123      # Create PR
gptcode issue ci 42         # Handle CI failures
gptcode issue review 42     # Address review comments

Limitations:

✅ Test Execution & Validation (38%)

GPTCode can run and validate code across multiple languages:

Test Runners:

Linters:

Additional Validation:

Limitations:

✅ Error Recovery (80%)

GPTCode can automatically fix common failures:

How it works:

  1. Detects failure
  2. Analyzes error message and context
  3. Generates fix using LLM
  4. Applies fix and re-runs validation
  5. Retries up to 2 times

Success rate: ~70% for simple failures

Limitations:

✅ Codebase Understanding (60%)

GPTCode can analyze and navigate codebases:

Example:

Issue: "Add password validation with special characters"

GPTCode identifies:
1. [HIGH 0.9] auth/validator.go - Main validation logic
2. [MED 0.6] auth/validator_test.go - Needs test updates  
3. [LOW 0.3] config/security.go - May need config

Limitations:

What GPTCode Cannot Do (Yet)

✅ Complex Code Modifications (10/12 scenarios)

Implemented:

Not yet implemented:

Examples:

gptcode gen migration "add user email"
# Detects model changes
# Generates SQL with up/down migrations

gptcode refactor api
# Scans routes in handlers/controllers
# Generates/updates handler functions
# Creates/updates corresponding tests

gptcode refactor signature processData "(ctx context.Context, data []byte) error"
# Finds function definition
# Updates all call sites across files
# Preserves functionality

gptcode refactor breaking
# Detects breaking changes via git diff
# Finds all consumers (functions/types)
# Generates migration plan
# Updates consuming code automatically

gptcode security scan
# Scans vulnerabilities (govulncheck, npm audit, safety, bundle audit)
# Reports severity and CVEs

gptcode security scan --fix
# Auto-updates dependencies
# LLM fixes code if needed

gptcode evolve generate "add email column to users"
# Generates multi-phase migration strategy
# Phase 1: Add nullable column
# Phase 2: Backfill data
# Phase 3: Add NOT NULL constraint
# Includes rollback for each phase

Limitations:

Why others not implemented: These require deep architectural understanding and multi-step coordination. Coming in future releases.

✅ Test Generation (8/8 scenarios) - 100% COMPLETE

Implemented:

Example:

gptcode gen test pkg/calculator/calculator.go
# Generates: pkg/calculator/calculator_test.go
# Validates: Compiles and runs

Limitations:

🟡 Merge Conflicts (3/5 scenarios)

Implemented:

Not yet implemented:

Examples:

gptcode merge resolve
# Detects all conflicted files
# Uses LLM to resolve each conflict
# Validates resolution (no conflict markers)
# Stages resolved files

Limitations: AI-powered conflict resolution using LLM - always review resolved conflicts before committing.

✅ Advanced Git Operations (5/5 scenarios) - 100% COMPLETE

Implemented:

Examples:

gptcode git bisect v1.0.0 HEAD
# Automatically runs tests on each commit
# Finds which commit introduced the bug
# Provides LLM analysis of the breaking commit

gptcode git cherry-pick abc123 def456
# Applies commits with automatic conflict resolution
# Uses LLM to resolve conflicts intelligently

gptcode git rebase main
# Rebases with AI-powered conflict resolution
# Continues automatically after resolving

gptcode git squash HEAD~3
# Squashes last 3 commits into one
# Generates intelligent commit message via LLM

gptcode git reword HEAD
# Suggests improved commit message
# Follows best practices (subject + body)

Limitations:

✅ Documentation Updates (3/3 scenarios) - 100% COMPLETE

Implemented:

Examples:

gptcode gen changelog           # All commits since last tag
gptcode docs update             # Analyze and preview README updates
gptcode docs update --apply     # Apply updates automatically

Limitations:

Workaround: Use gptcode chat mode to draft API documentation.

Roadmap

Next Release (Targeting 80% Autonomy)

Phase 7: Complex Code Modifications (10 remaining scenarios)

Phase 8: Test Generation (1 remaining scenario)

Phase 9: Documentation (1 remaining scenario)

How to Check Current Status

Run E2E tests to see what’s working:

# All tests
go test -tags=e2e ./tests/e2e/... -v

# Specific capability
go test -tags=e2e ./tests/e2e/run -run TestGitHubIssueIntegration -v

Skipped tests (t.Skip()) represent features not yet implemented.

Version History

v0.x (December 2025) - 100% MVAA

Future Releases

Track progress at: https://github.com/jadercorrea/gptcode/milestones

Reporting Issues

Found a limitation not listed here? Open an issue

See something marked as “not working” that actually works for you? Let us know!