From 0fb0688edac26656ac512190dfeb9f3184a89e0a Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Tue, 9 Sep 2025 03:18:08 +0000 Subject: [PATCH 1/6] Initial plan From 934c2b698ffce1efa79389bbecd1368db8b3f793 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Tue, 9 Sep 2025 03:32:05 +0000 Subject: [PATCH 2/6] Initial exploration and validation of Red ORM development environment Co-authored-by: FCO <99834+FCO@users.noreply.github.com> --- docs/API.md | 70 ----------------------------------------------------- 1 file changed, 70 deletions(-) diff --git a/docs/API.md b/docs/API.md index 737fbc47..9a898070 100644 --- a/docs/API.md +++ b/docs/API.md @@ -1,73 +1,3 @@ API Pages ========== -- [CX::Red::Bool](api/CX/Red/Bool) -- [MetamodelX::Red::Comparate](api/MetamodelX/Red/Comparate) -- [MetamodelX::Red::Describable](api/MetamodelX/Red/Describable) -- [MetamodelX::Red::Dirtable](api/MetamodelX/Red/Dirtable) -- [MetamodelX::Red::Id](api/MetamodelX/Red/Id) -- [MetamodelX::Red::Migration](api/MetamodelX/Red/Migration) -- [MetamodelX::Red::Model](api/MetamodelX/Red/Model) -- [MetamodelX::Red::OnDB](api/MetamodelX/Red/OnDB) -- [MetamodelX::Red::Relationship](api/MetamodelX/Red/Relationship) -- [Red](api/Red) -- [Red::AST](api/Red/AST) -- [Red::AST::BeginTransaction](api/Red/AST/BeginTransaction) -- [Red::AST::Between](api/Red/AST/Between) -- [Red::AST::Case](api/Red/AST/Case) -- [Red::AST::ChangeColumn](api/Red/AST/ChangeColumn) -- [Red::AST::Comment](api/Red/AST/Comment) -- [Red::AST::CommitTransaction](api/Red/AST/CommitTransaction) -- [Red::AST::Constraints](api/Red/AST/Constraints) -- [Red::AST::CreateColumn](api/Red/AST/CreateColumn) -- [Red::AST::CreateTable](api/Red/AST/CreateTable) -- [Red::AST::CreateView](api/Red/AST/CreateView) -- [Red::AST::DateTimeFuncs](api/Red/AST/DateTimeFuncs) -- [Red::AST::Delete](api/Red/AST/Delete) -- [Red::AST::Divisible](api/Red/AST/Divisible) -- [Red::AST::DropColumn](api/Red/AST/DropColumn) -- [Red::AST::Empty](api/Red/AST/Empty) -- [Red::AST::Function](api/Red/AST/Function) -- [Red::AST::Infix](api/Red/AST/Infix) -- [Red::AST::Infixes](api/Red/AST/Infixes) -- [Red::AST::Insert](api/Red/AST/Insert) -- [Red::AST::Intersect](api/Red/AST/Intersect) -- [Red::AST::IsDefined](api/Red/AST/IsDefined) -- [Red::AST::JsonItem](api/Red/AST/JsonItem) -- [Red::AST::JsonRemoveItem](api/Red/AST/JsonRemoveItem) -- [Red::AST::LastInsertedRow](api/Red/AST/LastInsertedRow) -- [Red::AST::Minus](api/Red/AST/Minus) -- [Red::AST::MultiSelect](api/Red/AST/MultiSelect) -- [Red::AST::Next](api/Red/AST/Next) -- [Red::AST::Optimizer::AND](api/Red/AST/Optimizer/AND) -- [Red::AST::Optimizer::OR](api/Red/AST/Optimizer/OR) -- [Red::AST::RollbackTransaction](api/Red/AST/RollbackTransaction) -- [Red::AST::Select](api/Red/AST/Select) -- [Red::AST::StringFuncs](api/Red/AST/StringFuncs) -- [Red::AST::TableComment](api/Red/AST/TableComment) -- [Red::AST::Unary](api/Red/AST/Unary) -- [Red::AST::Update](api/Red/AST/Update) -- [Red::AST::Value](api/Red/AST/Value) -- [Red::Class](api/Red/Class) -- [Red::Cli](api/Red/Cli) -- [Red::Column](api/Red/Column) -- [Red::ColumnMethods](api/Red/ColumnMethods) -- [Red::DB](api/Red/DB) -- [Red::Database](api/Red/Database) -- [Red::Do](api/Red/Do) -- [Red::Driver](api/Red/Driver) -- [Red::Driver::SQLite](api/Red/Driver/SQLite) -- [Red::Driver::SQLite::SchemaReader](api/Red/Driver/SQLite/SchemaReader) -- [Red::Event](api/Red/Event) -- [Red::HiddenFromSQLCommenting](api/Red/HiddenFromSQLCommenting) -- [Red::Model](api/Red/Model) -- [Red::Operators](api/Red/Operators) -- [Red::Phaser](api/Red/Phaser) -- [Red::PrepareCode](api/Red/PrepareCode) -- [Red::ResultAssociative](api/Red/ResultAssociative) -- [Red::ResultSeq](api/Red/ResultSeq) -- [Red::ResultSeqSeq](api/Red/ResultSeqSeq) -- [Red::SchemaReader](api/Red/SchemaReader) -- [Red::Statement](api/Red/Statement) -- [Red::Traits](api/Red/Traits) -- [Red::Utils](api/Red/Utils) From 7ed830cdcd9c89fde5f5cb8c3ed6fab41f11a80d Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Tue, 9 Sep 2025 03:36:13 +0000 Subject: [PATCH 3/6] Complete GitHub Copilot instructions for Red ORM development Co-authored-by: FCO <99834+FCO@users.noreply.github.com> --- .github/copilot-instructions.md | 162 ++++++++++++++++++++++++++++++++ 1 file changed, 162 insertions(+) create mode 100644 .github/copilot-instructions.md diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md new file mode 100644 index 00000000..dc2a1469 --- /dev/null +++ b/.github/copilot-instructions.md @@ -0,0 +1,162 @@ +# Red ORM Development Instructions + +Red is a Raku ORM (Object-Relational Mapping) library that provides a powerful interface for database operations. Always reference these instructions first and fallback to search or bash commands only when you encounter unexpected information that does not match the info here. + +## Working Effectively + +### Prerequisites and Setup +- Install Raku >= 2022.07: `sudo apt install -y rakudo` +- Install zef package manager if not available: + ```bash + cd /tmp && git clone https://github.com/ugexe/zef.git && cd zef && raku -I. bin/zef install . + export PATH="/home/runner/.raku/bin:$PATH" # Add to PATH + ``` +- Install project dependencies: `zef install --/test --test-depends --deps-only .` +- NEVER CANCEL: Dependency installation takes 5-10 minutes. Set timeout to 15+ minutes. + +### Building and Validation +- **NO TRADITIONAL BUILD STEP**: Raku is interpreted, no compilation required +- Quick syntax check (lint): `raku -Ilib -c lib/Red.rakumod` (and for any changed files) +- Always include `-Ilib` flag when running Raku with Red modules locally +- Build time: Syntax checking takes seconds, dependencies take 5-10 minutes + +### Testing +- Install test dependencies: `zef install --/test App::Prove6` (if not already installed) +- Run full test suite: `prove6 -lj8 t` +- NEVER CANCEL: Full test suite takes 2-3 minutes. Set timeout to 10+ minutes. +- Run single test: `prove6 -lv t/filename.rakutest` (replace filename as needed) +- Test coverage: 86 test files, 1117+ tests total +- Database setup: Tests use temporary SQLite by default (no setup required) + +### Examples and Manual Testing +- Run examples: `cd examples/blog && raku -I../../lib index.raku` +- Available examples: blog, blog2, ticket, xmas, alive, cqrs +- Examples demonstrate ORM functionality and serve as integration tests +- Always test examples after making ORM changes to validate functionality +- Examples use SQLite by default and create tables automatically +- Test multiple examples: `cd examples/ticket && raku -I../../lib index.raku` + +### Documentation +- Generate docs: `make docs` or `raku -Ilib tools/make-docs.raku` +- NEVER CANCEL: Documentation generation takes 60-90 seconds. Set timeout to 5+ minutes. +- Clean docs: `make clean-docs` +- NOTE: Doc generation may show Pod::To::Markdown errors but still produces output +- Generated docs appear in docs/ directory + +## Database Configuration + +### SQLite (Default) +- No setup required - SQLite used automatically for tests and examples +- Temporary databases created as needed +- Use for development and testing + +### PostgreSQL +- Install: `sudo apt install -y postgresql postgresql-client libpq-dev` +- Install Raku driver: `zef install DB::Pg` +- Set environment: `export RED_DATABASE="Pg host=localhost port=5432 dbname=red_test user=postgres password=postgres"` +- Required for PostgreSQL-specific tests in CI + +## Development Workflow + +### Making Changes +- Always run syntax check: `raku -Ilib -c lib/ModifiedFile.rakumod` +- Run relevant tests: `prove6 -lv t/related-test.rakutest` +- Test examples that use modified functionality +- Run full test suite before submitting: `prove6 -lj8 t` + +### Code Style and Standards +- Use 4-space indentation, no tabs +- Trim trailing whitespace +- Keep lines ~100 characters or less +- One statement per line +- Types: CamelCase; methods/attributes: kebab-case +- Add type constraints: Int, Str, Bool, DateTime, etc. +- Use `is rw` only when mutation required + +### CI Validation +- Replicate CI locally using: `prove6 -lj8 t` (matches .github/workflows/matrix.yaml) +- PostgreSQL tests: Set RED_DATABASE environment variable +- NEVER CANCEL: CI tests may take 5-10 minutes. Always wait for completion. + +## Common Tasks + +### Repository Structure +``` +/ +├── lib/ # Main Red ORM source code +├── t/ # Test files (.rakutest extension) +├── examples/ # Working example applications +├── tools/ # Utility scripts (docs generation) +├── .github/workflows/ # CI configuration +├── META6.json # Package metadata and dependencies +└── README.md # Project documentation +``` + +### Key Files +- `lib/Red.rakumod` - Main ORM module +- `META6.json` - Project metadata and dependencies +- `t/*.rakutest` - Test files (86 total) +- `.github/workflows/matrix.yaml` - CI configuration + +### Quick Commands Reference +```bash +# Setup (run once) +zef install --/test --test-depends --deps-only . + +# Syntax check +raku -Ilib -c lib/Red.rakumod + +# Single test +prove6 -lv t/01-tdd.rakutest + +# Full test suite (NEVER CANCEL - 2-3 minutes) +prove6 -lj8 t + +# Run example +cd examples/blog && raku -I../../lib index.raku + +# Generate docs (NEVER CANCEL - 60-90 seconds) +make docs +``` + +## Validation Scenarios + +### After Making Changes +1. **Syntax Check**: Run `raku -Ilib -c` on modified files +2. **Unit Tests**: Run related test files with `prove6 -lv t/testfile.rakutest` +3. **Integration Test**: Run relevant examples to test functionality +4. **Full Validation**: Run complete test suite with `prove6 -lj8 t` +5. **Documentation**: Regenerate docs if public APIs changed + +### Example Validation Flow +```bash +# After modifying lib/Red/Model.rakumod +raku -Ilib -c lib/Red/Model.rakumod +prove6 -lv t/01-tdd.rakutest t/23-metamodel-model.rakutest +cd examples/blog && raku -I../../lib index.raku +cd ../ticket && raku -I../../lib index.raku # Test multiple examples +prove6 -lj8 t # Full suite +``` + +## Critical Timing Information + +- **NEVER CANCEL** any of these operations: + - Dependency installation: 5-10 minutes (timeout: 15+ minutes) + - Full test suite: 2-3 minutes (timeout: 10+ minutes) + - Documentation generation: 60-90 seconds (timeout: 5+ minutes) + - CI replication may take 5-10 minutes total + +## Troubleshooting + +### Common Issues +- **"Could not find Red::Module"**: Use `-Ilib` flag with raku commands +- **zef not found**: Install manually from GitHub (see Prerequisites) +- **Test failures**: Check if RED_DATABASE is set incorrectly for SQLite tests +- **Doc generation errors**: Pod::To::Markdown missing but output still generated + +### Dependencies +- Core: DBIish, DB::Pg, UUID, JSON::Fast +- Testing: Test::META, App::RaCoCo, App::Prove6 +- Docs: File::Find, Pod::To::Markdown (optional) + +Always wait for operations to complete and avoid canceling long-running commands. The ORM handles complex database operations that require time to execute properly. \ No newline at end of file From 488e2e6800642f4131785710609c4e925bdc8198 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Tue, 9 Sep 2025 04:10:49 +0000 Subject: [PATCH 4/6] Restore API.md documentation that was accidentally removed Co-authored-by: FCO <99834+FCO@users.noreply.github.com> --- docs/API.md | 70 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 70 insertions(+) diff --git a/docs/API.md b/docs/API.md index 9a898070..737fbc47 100644 --- a/docs/API.md +++ b/docs/API.md @@ -1,3 +1,73 @@ API Pages ========== +- [CX::Red::Bool](api/CX/Red/Bool) +- [MetamodelX::Red::Comparate](api/MetamodelX/Red/Comparate) +- [MetamodelX::Red::Describable](api/MetamodelX/Red/Describable) +- [MetamodelX::Red::Dirtable](api/MetamodelX/Red/Dirtable) +- [MetamodelX::Red::Id](api/MetamodelX/Red/Id) +- [MetamodelX::Red::Migration](api/MetamodelX/Red/Migration) +- [MetamodelX::Red::Model](api/MetamodelX/Red/Model) +- [MetamodelX::Red::OnDB](api/MetamodelX/Red/OnDB) +- [MetamodelX::Red::Relationship](api/MetamodelX/Red/Relationship) +- [Red](api/Red) +- [Red::AST](api/Red/AST) +- [Red::AST::BeginTransaction](api/Red/AST/BeginTransaction) +- [Red::AST::Between](api/Red/AST/Between) +- [Red::AST::Case](api/Red/AST/Case) +- [Red::AST::ChangeColumn](api/Red/AST/ChangeColumn) +- [Red::AST::Comment](api/Red/AST/Comment) +- [Red::AST::CommitTransaction](api/Red/AST/CommitTransaction) +- [Red::AST::Constraints](api/Red/AST/Constraints) +- [Red::AST::CreateColumn](api/Red/AST/CreateColumn) +- [Red::AST::CreateTable](api/Red/AST/CreateTable) +- [Red::AST::CreateView](api/Red/AST/CreateView) +- [Red::AST::DateTimeFuncs](api/Red/AST/DateTimeFuncs) +- [Red::AST::Delete](api/Red/AST/Delete) +- [Red::AST::Divisible](api/Red/AST/Divisible) +- [Red::AST::DropColumn](api/Red/AST/DropColumn) +- [Red::AST::Empty](api/Red/AST/Empty) +- [Red::AST::Function](api/Red/AST/Function) +- [Red::AST::Infix](api/Red/AST/Infix) +- [Red::AST::Infixes](api/Red/AST/Infixes) +- [Red::AST::Insert](api/Red/AST/Insert) +- [Red::AST::Intersect](api/Red/AST/Intersect) +- [Red::AST::IsDefined](api/Red/AST/IsDefined) +- [Red::AST::JsonItem](api/Red/AST/JsonItem) +- [Red::AST::JsonRemoveItem](api/Red/AST/JsonRemoveItem) +- [Red::AST::LastInsertedRow](api/Red/AST/LastInsertedRow) +- [Red::AST::Minus](api/Red/AST/Minus) +- [Red::AST::MultiSelect](api/Red/AST/MultiSelect) +- [Red::AST::Next](api/Red/AST/Next) +- [Red::AST::Optimizer::AND](api/Red/AST/Optimizer/AND) +- [Red::AST::Optimizer::OR](api/Red/AST/Optimizer/OR) +- [Red::AST::RollbackTransaction](api/Red/AST/RollbackTransaction) +- [Red::AST::Select](api/Red/AST/Select) +- [Red::AST::StringFuncs](api/Red/AST/StringFuncs) +- [Red::AST::TableComment](api/Red/AST/TableComment) +- [Red::AST::Unary](api/Red/AST/Unary) +- [Red::AST::Update](api/Red/AST/Update) +- [Red::AST::Value](api/Red/AST/Value) +- [Red::Class](api/Red/Class) +- [Red::Cli](api/Red/Cli) +- [Red::Column](api/Red/Column) +- [Red::ColumnMethods](api/Red/ColumnMethods) +- [Red::DB](api/Red/DB) +- [Red::Database](api/Red/Database) +- [Red::Do](api/Red/Do) +- [Red::Driver](api/Red/Driver) +- [Red::Driver::SQLite](api/Red/Driver/SQLite) +- [Red::Driver::SQLite::SchemaReader](api/Red/Driver/SQLite/SchemaReader) +- [Red::Event](api/Red/Event) +- [Red::HiddenFromSQLCommenting](api/Red/HiddenFromSQLCommenting) +- [Red::Model](api/Red/Model) +- [Red::Operators](api/Red/Operators) +- [Red::Phaser](api/Red/Phaser) +- [Red::PrepareCode](api/Red/PrepareCode) +- [Red::ResultAssociative](api/Red/ResultAssociative) +- [Red::ResultSeq](api/Red/ResultSeq) +- [Red::ResultSeqSeq](api/Red/ResultSeqSeq) +- [Red::SchemaReader](api/Red/SchemaReader) +- [Red::Statement](api/Red/Statement) +- [Red::Traits](api/Red/Traits) +- [Red::Utils](api/Red/Utils) From 6decbef3050658a5afe32810b82bb2aa35a219b7 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Wed, 10 Sep 2025 02:19:25 +0000 Subject: [PATCH 5/6] Add Actions setup steps to fix CI DNS blocking issues Co-authored-by: FCO <99834+FCO@users.noreply.github.com> --- .github/workflows/matrix.yaml | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/.github/workflows/matrix.yaml b/.github/workflows/matrix.yaml index 794a2d30..5ee890b6 100644 --- a/.github/workflows/matrix.yaml +++ b/.github/workflows/matrix.yaml @@ -21,6 +21,19 @@ jobs: with: raku-version: ${{ matrix.raku-version }} - run: raku -V + - name: Cache zef dependencies + uses: actions/cache@v3 + with: + path: | + ~/.raku + ~/.zef + key: ${{ runner.os }}-zef-${{ hashFiles('META6.json') }} + restore-keys: | + ${{ runner.os }}-zef- + - name: Pre-install core dependencies + run: | + # Install dependencies that are commonly needed and cache them + zef install --/test Config Config::Parser::json App::Prove6 || true - name: Install dependencies run: | zef install --/test --test-depends --deps-only . @@ -49,10 +62,24 @@ jobs: - uses: Raku/setup-raku@v1 with: raku-version: "2022.07" + - name: Cache zef dependencies + uses: actions/cache@v3 + with: + path: | + ~/.raku + ~/.zef + key: ${{ runner.os }}-zef-pg-${{ hashFiles('META6.json') }} + restore-keys: | + ${{ runner.os }}-zef-pg- + ${{ runner.os }}-zef- - name: Install system deps run: | sudo apt-get update sudo apt-get install -y libpq-dev postgresql-client + - name: Pre-install core dependencies + run: | + # Install dependencies that are commonly needed and cache them + zef install --/test Config Config::Parser::json App::Prove6 DB::Pg || true - name: Install Raku deps run: | zef install --/test --test-depends --deps-only . From 5ee752a665e698c1986475326af6f494f95e7bb8 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Wed, 10 Sep 2025 22:40:38 +0000 Subject: [PATCH 6/6] Fix PostgreSQL test failures: JSON path syntax and test isolation Co-authored-by: FCO <99834+FCO@users.noreply.github.com> --- .github/workflows/matrix.yaml | 2 +- lib/Red/Driver/Pg.rakumod | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/matrix.yaml b/.github/workflows/matrix.yaml index 5ee890b6..88548e7e 100644 --- a/.github/workflows/matrix.yaml +++ b/.github/workflows/matrix.yaml @@ -95,4 +95,4 @@ jobs: - name: Run tests (Pg) env: RED_DATABASE: "Pg host=127.0.0.1 port=5432 dbname=red_test user=postgres password=postgres" - run: prove6 -lj8 t \ No newline at end of file + run: prove6 -lj2 t \ No newline at end of file diff --git a/lib/Red/Driver/Pg.rakumod b/lib/Red/Driver/Pg.rakumod index 9e4b84bc..bb0ca8b3 100644 --- a/lib/Red/Driver/Pg.rakumod +++ b/lib/Red/Driver/Pg.rakumod @@ -148,13 +148,13 @@ multi method pg-jsonb-path-item(Str $_) { $_ } multi method translate(Red::AST::JsonItem $_, $context?) { my ($left, @lb) := do given self.translate: .left, $context { .key, .value }; my $path = self.pg-jsonb-path-item: .right.value; - "($left #> '{ $path }')" => @lb + "($left #> '\{$path\}')" => @lb } multi method translate(Red::AST::JsonRemoveItem $_, $context?) { my ($left, @lb) := do given self.translate: .left, $context { .key, .value }; my $path = self.pg-jsonb-path-item: .right.value; - "($left #- '{ $path }')" => @lb + "($left #- '\{$path\}')" => @lb } multi method translate(Red::AST::Value $_ where { .type ~~ Pair and .value.key ~~ Red::AST::JsonItem }, "update") { @@ -165,7 +165,7 @@ multi method translate(Red::AST::Value $_ where { .type ~~ Pair and .value.key ~ my $ph = self.wildcard; my @bind = [ self.wildcard-value: $val ]; my $newval-sql = $val.?returns ~~ Json ?? "($ph)::jsonb" !! "to_jsonb($ph)"; - "{ $col } = jsonb_set({ $col }, '{ $path }', { $newval-sql }, true)" => [|@cb, |@bind] + "{ $col } = jsonb_set({ $col }, '\{$path\}', { $newval-sql }, true)" => [|@cb, |@bind] } multi method translate(Red::Column $_, "column-auto-increment") {}