Add installable migrations system for easier upgrades

README.md

@@ -22,6 +22,9 @@
   - [Installation](#installation)
   - [Quick Setup](#quick-setup)
   - [Basic Configuration](#basic-configuration)
+- [Upgrading](#upgrading)
+  - [Upgrading from Schema-based Installations](#upgrading-from-schema-based-installations)
+  - [Migration-based Upgrades](#migration-based-upgrades)
 - [Authentication](#authentication)
   - [Authentication Setup](#authentication-setup)
   - [Authentication Examples](#authentication-examples)
@@ -206,6 +209,93 @@ RailsPulse.configure do |config|
 end
 ```
 
+## Upgrading
+
+Rails Pulse uses a **migration-based upgrade system** starting from version 0.2.5+ to ensure smooth schema updates without manual intervention.
+
+### Migration-based Upgrades (Recommended)
+
+**For new installations (v0.2.5+)**, the installation process uses standard Rails migrations by default:
+
+```bash
+# Update your Gemfile
+bundle update rails_pulse
+
+# Generate and run any new migrations
+rails generate rails_pulse:upgrade
+rails db:migrate
+
+# Restart your server
+```
+
+The `rails_pulse:upgrade` generator intelligently detects missing schema changes and generates only the necessary migrations.
+
+### Upgrading from Schema-based Installations
+
+**If you installed Rails Pulse before v0.2.5** (using the schema file approach), you may encounter issues when upgrading because the schema file approach couldn't add new columns to existing tables.
+
+#### Common Upgrade Issue
+
+When upgrading from v0.2.3 or earlier to v0.2.4+, you may see this error:
+
+```
+NameError (undefined local variable or method 'tags' for an instance of RailsPulse::Request)
+```
+
+This occurs because v0.2.4 added the tagging feature but couldn't automatically add the `tags` columns to existing installations.
+
+#### Solution
+
+Run the upgrade generator to add missing columns:
+
+```bash
+# This will detect and generate migrations for any missing columns/tables
+rails generate rails_pulse:upgrade
+rails db:migrate
+
+# Restart your server
+```
+
+The upgrade generator checks your current schema and creates migrations only for what's missing:
+- **Tags columns** (added in v0.2.4) - for routes, queries, and requests tables
+- **Query analysis columns** (added in v0.2.x) - for enhanced query performance insights
+- **Summaries table** (added in v0.2.x) - for aggregated performance metrics
+
+#### Manual Upgrade (Alternative)
+
+If you prefer to see exactly what's being added, you can manually create a migration:
+
+```ruby
+# db/migrate/TIMESTAMP_add_missing_rails_pulse_columns.rb
+class AddMissingRailsPulseColumns < ActiveRecord::Migration[7.1]
+  def change
+    # Add tags columns if missing
+    add_column :rails_pulse_routes, :tags, :text unless column_exists?(:rails_pulse_routes, :tags)
+    add_column :rails_pulse_queries, :tags, :text unless column_exists?(:rails_pulse_queries, :tags)
+    add_column :rails_pulse_requests, :tags, :text unless column_exists?(:rails_pulse_requests, :tags)
+  end
+end
+```
+
+Then run: `rails db:migrate`
+
+### Best Practices
+
+1. **Always run the upgrade generator** after updating Rails Pulse
+2. **Review generated migrations** before running `db:migrate` in production
+3. **Test in development/staging** before deploying to production
+4. **Check the CHANGELOG** for breaking changes or new features
+
+### Future Upgrades
+
+Starting from v0.2.5+, all schema changes will include proper migrations, making upgrades seamless:
+
+```bash
+bundle update rails_pulse
+rails generate rails_pulse:upgrade  # Only if new schema changes exist
+rails db:migrate
+```
+
 ## Authentication
 
 Rails Pulse supports flexible authentication to secure access to your monitoring dashboard.

lib/generators/rails_pulse/install_generator.rb

@@ -4,124 +4,80 @@ class InstallGenerator < Rails::Generators::Base
       include Rails::Generators::Migration
       source_root File.expand_path("templates", __dir__)
 
-      desc "Install Rails Pulse with flexible database setup options"
+      desc "Install Rails Pulse (recommended: use migrations for easier upgrades)"
 
-      class_option :database, type: :string, default: "single",
-                   desc: "Database setup: 'single' (default) or 'separate'"
+      class_option :use_schema, type: :boolean, default: false,
+                   desc: "Use schema file instead of migrations (legacy)"
 
-      def self.next_migration_number(path)
-        next_migration_number = current_migration_number(path) + 1
+      def self.next_migration_number(dirname)
+        next_migration_number = current_migration_number(dirname) + 1
         ActiveRecord::Migration.next_migration_number(next_migration_number)
       end
 
-      def copy_schema
-        copy_file "db/rails_pulse_schema.rb", "db/rails_pulse_schema.rb"
-      end
-
-      def create_migration_directory
-        create_file "db/rails_pulse_migrate/.keep"
-      end
-
-      def copy_gem_migrations
-        gem_migrations_path = File.expand_path("../../../db/rails_pulse_migrate", __dir__)
-        destination_dir = separate_database? ? "db/rails_pulse_migrate" : "db/migrate"
-
-        if File.directory?(gem_migrations_path)
-          Dir.glob("#{gem_migrations_path}/*.rb").each do |migration_file|
-            migration_name = File.basename(migration_file)
-            destination_path = File.join(destination_dir, migration_name)
-
-            # Only copy if it doesn't already exist in the destination
-            # Use File.join with destination_root to check the actual location
-            full_destination_path = File.join(destination_root, destination_path)
-            unless File.exist?(full_destination_path)
-              copy_file migration_file, destination_path
-            end
-          end
+      def install_database_tables
+        if use_schema?
+          install_via_schema
+        else
+          install_via_migrations
         end
       end
 
       def copy_initializer
         copy_file "rails_pulse.rb", "config/initializers/rails_pulse.rb"
       end
 
-      def setup_database_configuration
-        if separate_database?
-          create_separate_database_setup
-        else
-          create_single_database_setup
-        end
-      end
-
       def display_post_install_message
-        if separate_database?
-          display_separate_database_message
+        if use_schema?
+          display_schema_message
         else
-          display_single_database_message
+          display_migrations_message
         end
       end
 
       private
 
-      def separate_database?
-        options[:database] == "separate"
+      def use_schema?
+        options[:use_schema]
       end
 
-      def create_separate_database_setup
-        say "Setting up separate database configuration...", :green
-
-        # Migration directory already created by create_migration_directory
-        # Could add database.yml configuration here if needed
-        # For now, users will configure manually
+      def install_via_schema
+        say "Installing Rails Pulse using schema file (legacy)...", :yellow
+        copy_file "db/rails_pulse_schema.rb", "db/rails_pulse_schema.rb"
       end
 
-      def create_single_database_setup
-        say "Setting up single database configuration...", :green
-
-        # Create a migration that loads the schema
-        migration_template(
-          "migrations/install_rails_pulse_tables.rb",
-          "db/migrate/install_rails_pulse_tables.rb"
-        )
+      def install_via_migrations
+        say "Installing Rails Pulse using migrations (recommended)...", :green
+        migration_template "migrations/create_rails_pulse_tables.rb",
+                          "db/migrate/create_rails_pulse_tables.rb"
       end
 
-      def display_separate_database_message
+      def display_schema_message
         say <<~MESSAGE
 
-          Rails Pulse installation complete! (Separate Database Setup)
+          Rails Pulse installation complete! (Schema-based setup - Legacy)
 
           Next steps:
-          1. Add Rails Pulse database configuration to config/database.yml:
-
-             #{Rails.env}:
-               rails_pulse:
-                 <<: *default
-                 database: storage/#{Rails.env}_rails_pulse.sqlite3
-                 migrations_paths: db/rails_pulse_migrate
-
-          2. Run: rails db:prepare (creates database and loads schema)
-          3. Restart your Rails server
+          1. Run: rails db:prepare (creates database and loads schema)
+          2. Restart your Rails server
 
-          The schema file db/rails_pulse_schema.rb is your single source of truth.
-          Future upgrades will automatically copy new migrations to db/rails_pulse_migrate/
+          Note: Future upgrades may require manual schema synchronization.
+          Consider using migrations-based installation for easier upgrades:
+            rails generate rails_pulse:install
 
         MESSAGE
       end
 
-      def display_single_database_message
+      def display_migrations_message
         say <<~MESSAGE
 
-          Rails Pulse installation complete! (Single Database Setup)
+          Rails Pulse installation complete!
 
           Next steps:
-          1. Run: rails db:migrate (creates Rails Pulse tables in your main database)
+          1. Run: rails db:migrate
           2. Restart your Rails server
 
-          The schema file db/rails_pulse_schema.rb is your single source of truth.
-          Future upgrades will automatically copy new migrations to db/migrate/
-
-          Note: The installation migration loads from db/rails_pulse_schema.rb
-          and includes all current Rails Pulse tables and columns.
+          For upgrades, run: rails generate rails_pulse:upgrade
+          This will detect and generate any necessary schema updates.
 
         MESSAGE
       end

lib/generators/rails_pulse/migrations/create_rails_pulse_tables.rb

@@ -0,0 +1,112 @@
+class CreateRailsPulseTables < ActiveRecord::Migration[<%= ActiveRecord::Migration.current_version %>]
+  def change
+    # Routes table
+    create_table :rails_pulse_routes do |t|
+      t.string :method, null: false, comment: "HTTP method (e.g., GET, POST)"
+      t.string :path, null: false, comment: "Request path (e.g., /posts/index)"
+      t.text :tags, comment: "JSON array of tags for filtering and categorization"
+      t.timestamps
+    end
+
+    add_index :rails_pulse_routes, [ :method, :path ], unique: true, name: "index_rails_pulse_routes_on_method_and_path"
+
+    # Queries table
+    create_table :rails_pulse_queries do |t|
+      t.string :normalized_sql, limit: 1000, null: false, comment: "Normalized SQL query string (e.g., SELECT * FROM users WHERE id = ?)"
+      t.datetime :analyzed_at, comment: "When query analysis was last performed"
+      t.text :explain_plan, comment: "EXPLAIN output from actual SQL execution"
+      t.text :issues, comment: "JSON array of detected performance issues"
+      t.text :metadata, comment: "JSON object containing query complexity metrics"
+      t.text :query_stats, comment: "JSON object with query characteristics analysis"
+      t.text :backtrace_analysis, comment: "JSON object with call chain and N+1 detection"
+      t.text :index_recommendations, comment: "JSON array of database index recommendations"
+      t.text :n_plus_one_analysis, comment: "JSON object with enhanced N+1 query detection results"
+      t.text :suggestions, comment: "JSON array of optimization recommendations"
+      t.text :tags, comment: "JSON array of tags for filtering and categorization"
+      t.timestamps
+    end
+
+    add_index :rails_pulse_queries, :normalized_sql, unique: true, name: "index_rails_pulse_queries_on_normalized_sql", length: 191
+
+    # Requests table
+    create_table :rails_pulse_requests do |t|
+      t.references :route, null: false, foreign_key: { to_table: :rails_pulse_routes }, comment: "Link to the route"
+      t.decimal :duration, precision: 15, scale: 6, null: false, comment: "Total request duration in milliseconds"
+      t.integer :status, null: false, comment: "HTTP status code (e.g., 200, 500)"
+      t.boolean :is_error, null: false, default: false, comment: "True if status >= 500"
+      t.string :request_uuid, null: false, comment: "Unique identifier for the request (e.g., UUID)"
+      t.string :controller_action, comment: "Controller and action handling the request (e.g., PostsController#show)"
+      t.timestamp :occurred_at, null: false, comment: "When the request started"
+      t.text :tags, comment: "JSON array of tags for filtering and categorization"
+      t.timestamps
+    end
+
+    add_index :rails_pulse_requests, :occurred_at, name: "index_rails_pulse_requests_on_occurred_at"
+    add_index :rails_pulse_requests, :request_uuid, unique: true, name: "index_rails_pulse_requests_on_request_uuid"
+    add_index :rails_pulse_requests, [ :route_id, :occurred_at ], name: "index_rails_pulse_requests_on_route_id_and_occurred_at"
+
+    # Operations table
+    create_table :rails_pulse_operations do |t|
+      t.references :request, null: false, foreign_key: { to_table: :rails_pulse_requests }, comment: "Link to the request"
+      t.references :query, foreign_key: { to_table: :rails_pulse_queries }, index: true, comment: "Link to the normalized SQL query"
+      t.string :operation_type, null: false, comment: "Type of operation (e.g., database, view, gem_call)"
+      t.string :label, null: false, comment: "Descriptive name (e.g., SELECT FROM users WHERE id = 1, render layout)"
+      t.decimal :duration, precision: 15, scale: 6, null: false, comment: "Operation duration in milliseconds"
+      t.string :codebase_location, comment: "File and line number (e.g., app/models/user.rb:25)"
+      t.float :start_time, null: false, default: 0.0, comment: "Operation start time in milliseconds"
+      t.timestamp :occurred_at, null: false, comment: "When the request started"
+      t.timestamps
+    end
+
+    add_index :rails_pulse_operations, :operation_type, name: "index_rails_pulse_operations_on_operation_type"
+    add_index :rails_pulse_operations, :occurred_at, name: "index_rails_pulse_operations_on_occurred_at"
+    add_index :rails_pulse_operations, [ :query_id, :occurred_at ], name: "index_rails_pulse_operations_on_query_and_time"
+    add_index :rails_pulse_operations, [ :query_id, :duration, :occurred_at ], name: "index_rails_pulse_operations_query_performance"
+    add_index :rails_pulse_operations, [ :occurred_at, :duration, :operation_type ], name: "index_rails_pulse_operations_on_time_duration_type"
+
+    # Summaries table
+    create_table :rails_pulse_summaries do |t|
+      # Time fields
+      t.datetime :period_start, null: false, comment: "Start of the aggregation period"
+      t.datetime :period_end, null: false, comment: "End of the aggregation period"
+      t.string :period_type, null: false, comment: "Aggregation period type: hour, day, week, month"
+
+      # Polymorphic association to handle both routes and queries
+      t.references :summarizable, polymorphic: true, null: false, index: true, comment: "Link to Route or Query"
+
+      # Universal metrics
+      t.integer :count, default: 0, null: false, comment: "Total number of requests/operations"
+      t.float :avg_duration, comment: "Average duration in milliseconds"
+      t.float :min_duration, comment: "Minimum duration in milliseconds"
+      t.float :max_duration, comment: "Maximum duration in milliseconds"
+      t.float :p50_duration, comment: "50th percentile duration"
+      t.float :p95_duration, comment: "95th percentile duration"
+      t.float :p99_duration, comment: "99th percentile duration"
+      t.float :total_duration, comment: "Total duration in milliseconds"
+      t.float :stddev_duration, comment: "Standard deviation of duration"
+
+      # Request/Route specific metrics
+      t.integer :error_count, default: 0, comment: "Number of error responses (5xx)"
+      t.integer :success_count, default: 0, comment: "Number of successful responses"
+      t.integer :status_2xx, default: 0, comment: "Number of 2xx responses"
+      t.integer :status_3xx, default: 0, comment: "Number of 3xx responses"
+      t.integer :status_4xx, default: 0, comment: "Number of 4xx responses"
+      t.integer :status_5xx, default: 0, comment: "Number of 5xx responses"
+
+      t.timestamps
+    end
+
+    # Unique constraint and indexes for summaries
+    add_index :rails_pulse_summaries, [ :summarizable_type, :summarizable_id, :period_type, :period_start ],
+              unique: true,
+              name: "idx_pulse_summaries_unique"
+    add_index :rails_pulse_summaries, [ :period_type, :period_start ], name: "index_rails_pulse_summaries_on_period"
+    add_index :rails_pulse_summaries, :created_at, name: "index_rails_pulse_summaries_on_created_at"
+
+    # Additional aggregation indexes
+    add_index :rails_pulse_requests, [ :created_at, :route_id ], name: "idx_requests_for_aggregation"
+    add_index :rails_pulse_requests, :created_at, name: "idx_requests_created_at"
+    add_index :rails_pulse_operations, [ :created_at, :query_id ], name: "idx_operations_for_aggregation"
+    add_index :rails_pulse_operations, :created_at, name: "idx_operations_created_at"
+  end
+end