52 Commits

Author SHA1 Message Date
7e81e90113 添加异步请求指标收集功能,优化请求记录逻辑,支持批量更新状态码、字节数和延迟统计,提升性能和可扩展性。 2025-07-11 20:38:24 +08:00
ef2ab55fe6 优化引用来源统计逻辑,移除持久化存储,添加最后访问时间字段,提升仪表板展示信息 2025-07-11 20:08:00 +08:00
5418e89e3b 移除路径统计相关代码和数据存储,更新指标处理逻辑,调整引用来源统计的加载数量限制,以简化代码和提高性能。 2025-03-23 12:28:15 +08:00
10aef5e73e feat(metrics): 优化指标数据清理和持久化策略
- 调整指标清理机制,智能保留高频路径和引用来源统计数据
- 修改清理任务频率为15分钟,并立即执行首次清理
- 优化指标存储服务保存间隔为30分钟,减少IO操作
- 在清理和保存过程中添加内存使用情况日志
- 强制执行垃圾回收,减少内存占用
- 移除部分冗余的性能指标统计项目
2025-03-10 03:30:39 +08:00
22c0d2e301 feat(metrics): 优化指标数据处理和清理机制
- 在指标收集器中新增定期清理任务,自动清理无效的统计数据
- 修改 MetricsHandler 中的延迟分布处理,移除冗余日志输出
- 优化 MetricsStorage 的数据加载逻辑,限制加载的统计数据数量
- 新增延迟分布单独持久化存储,减少主指标文件的复杂性
- 改进数据加载和恢复的日志记录,提供更清晰的操作反馈
2025-03-09 13:21:35 +08:00
006fa9a172 refactor(metrics): 移除延迟分布统计的冗余日志输出
- 删除 RecordRequest 和 GetStats 方法中的详细日志记录
- 保留核心的延迟分布统计逻辑,提高代码简洁性
- 减少不必要的日志输出,优化性能和可读性
2025-03-09 12:16:44 +08:00
f2e1b8cbf5 feat(metrics): 统一延迟分布桶命名并优化日志记录
- 将延迟分布桶键名从 "<10ms" 和 ">1s" 改为 "lt10ms" 和 "gt1s"
- 在指标处理和收集过程中添加详细日志,帮助诊断数据处理流程
- 前端页面适配新的桶键名,并提供友好的显示转换
- 增强延迟分布数据处理的健壮性和可读性
2025-03-09 11:43:26 +08:00
2cb88a4f5e fix(metrics): 优化延迟统计和分布数据处理
- 修复延迟统计分布数据获取逻辑,增加更健壮的类型转换
- 确保延迟分布桶始终存在,即使没有数据
- 在处理器中为空分布数据添加默认初始化
- 优化指标收集器中的延迟桶初始化和数据获取方法
2025-03-09 11:33:47 +08:00
b6b77b03ed feat(metrics): 增强指标展示和统计功能
- 在指标结构中新增延迟统计、错误统计和引用来源统计字段
- 更新前端仪表盘,添加延迟、带宽、错误和引用来源统计卡片
- 优化指标收集器,支持引用来源和错误类型统计
- 在工具函数中新增字符串转整数解析方法
- 简化引用来源URL处理,提取域名信息
2025-03-09 11:24:46 +08:00
f0c806292b feat(metrics): 完善指标存储服务集成
- 在主程序中添加指标存储服务初始化和停止逻辑
- 更新指标收集器的保存方法,支持外部指标存储服务
- 优化静态文件服务路径,使用新的 web/dist 目录
- 调整静态文件处理逻辑,支持更灵活的路由
2025-03-09 11:00:39 +08:00
e67a3880f5 删除统计保存功能 2025-03-09 10:48:43 +08:00
a0cea8f5b8 feat(metrics): 增强指标收集和恢复日志记录
- 在指标初始化和加载过程中添加详细日志
- 改进指标恢复逻辑,支持从不同数据结构恢复统计信息
- 添加更多日志输出,帮助诊断指标加载问题
- 优化数据验证逻辑,允许小范围的统计数据误差
- 增加对未找到数据的日志提示
2025-03-09 10:41:24 +08:00
095b087fd8 refactor(metrics): 优化指标收集和内存管理
- 修改路径统计数据结构为指针类型,提高性能
- 使用原子操作方法替换旧的原子操作
- 添加内存释放和垃圾回收机制
- 限制路径统计数量,只保留前20个路径
- 优化指标保存和文件清理逻辑,减少内存使用
2025-03-09 10:35:00 +08:00
015aa6bc15 feat(metrics): 增加指标保存和管理功能
- 在配置中新增指标保存间隔和最大文件数量设置
- 实现定期自动保存指标数据到文件
- 添加指标文件自动清理机制
- 优化指标收集和存储逻辑
- 在Web界面添加指标设置选项卡
2025-03-09 05:41:22 +08:00
76b549aa90 refactor(metrics): Enhance path metrics sorting with secondary lexicographic ordering
- Update path metrics sorting to consider request count and path name
- Implement secondary sorting by path name when request counts are equal
- Improve deterministic ordering of metrics for consistent display
2025-02-17 08:36:05 +08:00
ed63121a00 refactor(metrics): Introduce JSON-friendly metrics serialization
- Create PathMetricsJSON struct for clean JSON serialization
- Add ToJSON method to convert PathMetrics to JSON-compatible representation
- Update metrics handlers and collectors to use new JSON-friendly type
- Simplify type conversion in SafePathMetrics utility function
- Improve metrics data transfer and serialization consistency
2025-02-17 08:01:29 +08:00
55d3a9cebc refactor(metrics): Update metrics collection with atomic operations and improved type safety
- Replace manual increment methods with atomic store operations in PathMetrics
- Enhance GetStats method to handle different metric types and convert to value slice
- Update RequestLog JSON tags for consistent naming
- Add SafePathMetrics conversion for mixed pointer and value slices
- Improve type safety and consistency in metrics tracking
2025-02-17 07:56:52 +08:00
d0d752712e refactor(metrics): Comprehensive metrics system redesign with improved performance tracking
- Refactor Collector struct to use more efficient atomic counters and data structures
- Implement robust bandwidth tracking with sliding window mechanism
- Enhance path metrics with atomic operations and improved type safety
- Add data consistency checker for metrics integrity
- Optimize request logging with a more memory-efficient queue
- Simplify metrics collection and reporting logic
- Improve frontend metrics display with more detailed status code visualization
2025-02-17 07:46:58 +08:00
ff24191146 feat(dashboard): Enhance configuration management with advanced validation and UX improvements
- Add comprehensive input validation for path mappings, extension maps, and fixed paths
- Implement confirmation dialogs for deletion of paths, extensions, and fixed paths
- Improve error handling with detailed toast messages
- Add dynamic dialog state management for better user experience
- Enhance extension map editing with edit and delete functionality
- Implement more robust configuration import validation
2025-02-17 07:12:13 +08:00
4b1c774509 refactor(metrics): Optimize atomic counter initialization and status code tracking
- Refactor counter initialization to use pointer creation for better type safety
- Simplify atomic counter handling in RecordRequest method
- Add status code statistics collection with atomic load operations
- Improve consistency in counter initialization across different metrics tracking
2025-02-17 06:59:49 +08:00
e9aad806f8 refactor(metrics): Improve atomic counter handling and type safety in metrics collection
- Optimize latency bucket counter initialization using pointer creation
- Add type checking when loading latency distribution metrics
- Simplify counter initialization and access in metrics collector
- Enhance type safety for atomic counter operations
2025-02-17 06:36:38 +08:00
92910a608f feat(metrics): Add interval-based metrics tracking and reporting
- Implement time-interval statistics for requests, errors, and bytes
- Add per-second metrics calculation for requests and bandwidth
- Enhance GetStats method to return interval-specific performance data
- Track and reset interval status codes dynamically
- Improve metrics granularity with more detailed performance insights
2025-02-17 06:13:24 +08:00
5c3fb00d57 feat(dashboard): Add clickable links to paths in metrics tables
- Make top paths and recent requests table entries clickable
- Open links in new tabs with appropriate security attributes
- Add hover and color styles to improve link visibility and interaction
2025-02-16 20:46:41 +08:00
41f5b82661 feat(metrics): Enhance uptime formatting with more precise time display
- Update FormatUptime to include seconds in time representation
- Modify metrics handler to use new uptime formatting method
- Improve readability of uptime display with more granular time details
2025-02-16 20:24:31 +08:00
4e3a17ecd4 refactor(logging): Standardize logging format across components
- Update logging in cache, auth, proxy, metrics, and fixed path middleware
- Add consistent log message structure with method, path, status, and source
- Improve error logging with more descriptive and uniform messages
- Enhance log readability by using concise and informative log formats
2025-02-16 14:34:49 +08:00
a4c4688412 feat(cache): Add dynamic cache configuration management
- Implement GetConfig and UpdateConfig methods in CacheManager
- Add cache configuration endpoints in CacheAdminHandler
- Create frontend UI for dynamically updating cache settings
- Support configurable max age, cleanup interval, and cache size
- Add input validation for cache configuration updates
2025-02-16 13:57:12 +08:00
d00ab0a6e1 feat(metrics): Improve latency tracking and error visualization
- Add atomic operations to track minimum and maximum request latency
- Update GetStats method to handle uninitialized latency values
- Modify dashboard error chart with improved color scheme and legend
- Adjust chart margins and axis width for better readability
2025-02-15 18:24:55 +08:00
1c44fe1bf4 feat(metrics): Add status code tracking and improve path metrics handling
- Implement status code statistics tracking in Collector
- Update PathMetrics type to use pointer-based access
- Enhance atomic operations for request and error count tracking
- Modify type assertions to use pointer-based type conversion
2025-02-15 18:08:49 +08:00
c4cd99a827 feat(cache): Implement comprehensive caching system for proxy requests
- Add CacheManager to handle caching for proxy and mirror handlers
- Introduce cache-related API endpoints for stats, enabling, and clearing
- Modify proxy and mirror handlers to support caching GET requests
- Remove rate limiting and simplify request handling
- Update go.mod dependencies and remove unused imports
- Add cache hit/miss headers to responses
- Enhance metrics collection for cached requests
2025-02-15 17:00:16 +08:00
3e5950e3f6 feat(metrics,web): Enhance dashboard with advanced metrics visualization
- Integrate Recharts library for interactive data visualization
- Add detailed latency distribution charts and error type analysis
- Implement bandwidth and performance metrics graphs
- Update metrics collection to support more granular statistical tracking
- Modify frontend API routes to remove /admin prefix
- Improve metrics display with responsive and informative charts
2025-02-15 16:23:20 +08:00
f758db3531 refactor(metrics): Simplify metrics collection and improve performance
- Restructure Collector to use more efficient atomic operations
- Remove complex caching and monitoring dependencies
- Simplify path and status code tracking with sync.Map
- Optimize request logging with a dedicated request queue
- Remove unnecessary utility functions and pools
- Streamline stats generation and reduce complexity
2025-02-15 11:51:01 +08:00
33d6a51416 refactor(web): Migrate to modern web frontend and simplify admin routes
- Remove legacy static files, templates, and JavaScript
- Update main.go to serve SPA-style web application
- Modify admin route handling to support client-side routing
- Simplify configuration and metrics API endpoints
- Remove server-side template rendering in favor of static file serving
- Update Dockerfile and GitHub Actions to build web frontend
2025-02-15 11:44:09 +08:00
1f2f78dc64 refactor(metrics): remove historical data handling and streamline collector logic
- Eliminated the historical data storage and related functions from the metrics collector to simplify the codebase.
- Removed the historical data display section from the metrics template, enhancing clarity and focus on current metrics.
- Updated the CheckDataConsistency method to implement a simpler validation check for total errors against total requests.

These changes improve the maintainability of the metrics system by removing outdated features and ensuring a more straightforward data validation process.
2024-12-05 11:16:58 +08:00
058244cc70 feat(metrics): enhance configuration and performance monitoring
- Added new performance monitoring settings in the configuration, including maximum requests per minute and data transfer limits.
- Introduced validation parameters for metrics, ensuring data integrity and consistency checks during collection.
- Refactored the metrics collector to streamline initialization and improve error handling.
- Removed deprecated database-related code and optimized metrics saving processes.
- Enhanced historical data management with regular clean-up routines to maintain performance.

These changes improve the configurability and reliability of the metrics system, ensuring accurate data handling and enhanced monitoring capabilities.
2024-12-05 10:56:27 +08:00
f360e13ca3 feat(metrics): optimize stats collection and enhance error handling
- Refactored the GetStats method to improve memory management by utilizing an object pool for stats initialization.
- Enhanced error rate calculation and added average response time metrics for better performance insights.
- Streamlined path and referer statistics collection, ensuring only relevant data is processed and displayed.
- Improved sorting and selection logic for top paths and referers, enhancing the clarity of metrics reporting.
- Updated comments for better code readability and understanding of the metrics collection process.

These changes enhance the efficiency and accuracy of the metrics collector, providing more reliable performance data and insights.
2024-12-05 10:37:47 +08:00
14843301f7 feat(metrics): enhance initialization process in metrics collector
- Improved the initialization sequence by clearly defining steps for cache, monitoring, and object pool setup.
- Enabled Feishu webhook alerts for real-time notifications based on configuration.
- Ensured last save time is set during initialization to maintain accurate data persistence.
- Streamlined historical data loading process to enhance metrics accuracy.

These changes improve the organization and functionality of the metrics collector, ensuring better performance and alerting capabilities.
2024-12-05 10:33:55 +08:00
c71da4c1fe feat(metrics): initialize cache and object pool in metrics collector
- Added cache initialization to enhance performance and data retrieval efficiency.
- Introduced an object pool for managing statistics, optimizing memory usage during metrics processing.
- Improved code organization by clarifying initialization steps for cache and monitoring components.

These changes enhance the metrics collector's efficiency and resource management, ensuring better performance in data handling.
2024-12-05 10:30:10 +08:00
ef3bc0c5a6 feat(metrics): enhance metrics initialization and consistency checks
- Added initialization of last save time during collector setup to ensure timely data persistence.
- Implemented immediate saving of metrics after loading historical data to maintain accurate statistics.
- Improved error handling in SaveMetrics by validating input types before updating persistent stats.
- Modified CheckDataConsistency logic to allow for a tolerance of 5% in total request comparisons, enhancing reliability in metrics validation.

These changes improve the accuracy and reliability of the metrics collection process, ensuring better monitoring and data integrity.
2024-12-05 10:26:09 +08:00
1c22e4c020 栈溢出 2024-12-05 10:21:50 +08:00
e275326d91 feat(metrics): enhance configuration and metrics validation
- Added new configuration structures for loading, saving, and validation settings in MetricsConfig.
- Introduced constants for load retry counts, intervals, and validation parameters to improve configurability.
- Enhanced metrics collector with data validation and consistency checks during metrics loading.
- Implemented backup and restore functionality for metrics data, ensuring data integrity and recoverability.
- Improved logging for metrics operations, providing better insights into the metrics collection process.

These changes enhance the flexibility and reliability of the metrics system, ensuring accurate data handling and improved monitoring capabilities.
2024-12-05 10:15:20 +08:00
8c266a1ec2 feat(metrics): enhance metrics collection and monitoring capabilities
- Added new performance monitoring thresholds for maximum requests and data transfer rates.
- Introduced dynamic save interval adjustments based on data change rates to optimize database writes.
- Implemented a retry mechanism for loading recent statistics, improving reliability in data retrieval.
- Consolidated metrics saving into a single method, SaveAllMetrics, to streamline database transactions and improve performance.
- Enhanced error handling and logging throughout the metrics collection process for better monitoring.

These changes improve the efficiency and reliability of the metrics system, ensuring more accurate and timely data collection.
2024-12-05 09:51:43 +08:00
ad4211fd6f refactor(metrics): streamline recent stats loading and remove deprecated function
- Replaced the deprecated loadRecentStatusStats function with a new LoadRecentStats method in the MetricsDB, consolidating recent statistics loading for status codes, paths, and referers.
- Improved error handling and logging for loading recent stats, enhancing monitoring capabilities.
- This refactor simplifies the codebase and improves the efficiency of metrics collection, ensuring more accurate and timely data retrieval.
2024-12-05 09:35:11 +08:00
ba691431f4 状态码统计数组越界 2024-12-05 08:27:19 +08:00
864ebcc610 test 2024-12-05 08:23:38 +08:00
97a195c9dd feat(metrics): add recent status stats loading and optimize database transaction settings
- Implemented a new function to load recent status statistics from the database, enhancing the collector's ability to track real-time metrics.
- Added error logging for failed status stats loading to improve monitoring and debugging capabilities.
- Moved database transaction optimization settings outside of the transaction scope in SaveMetrics, improving performance during metric saving operations.
- Updated SQL queries in GetRecentMetrics to streamline time filtering logic, ensuring accurate retrieval of recent metrics.

These changes enhance the metrics collection process and improve the overall performance and reliability of the metrics dashboard.
2024-12-05 08:15:18 +08:00
e286fa9eaf feat(metrics): enhance metrics retention and improve data handling
- Introduced new constants for data retention periods, specifying retention for metrics, status codes, paths, and referers to improve data management.
- Updated the MetricsHistoryHandler to accept floating-point hours for more precise time queries, enhancing the flexibility of historical data retrieval.
- Enhanced the metrics dashboard layout with additional CSS styles for better visual presentation and user experience.
- Improved error handling in the response writing process to ensure robust logging of connection issues.
- Implemented a new function to check for connection closure errors, improving the reliability of the proxy handler.

These changes significantly enhance the metrics collection, retention, and user interface, providing a more robust framework for performance monitoring and data analysis.
2024-12-05 07:57:02 +08:00
26c945a3f9 feat(metrics): enhance metrics dashboard and data handling
- Improved the metrics dashboard layout with new styles for better visual presentation, including enhanced chart and control elements.
- Added functionality for auto-refreshing metrics and exporting data to CSV, improving user interaction and data accessibility.
- Implemented persistent statistics tracking in the collector, allowing for historical data retrieval and better performance monitoring.
- Enhanced database operations with optimizations for saving metrics and cleaning up old data, ensuring efficient data management.
- Introduced performance metrics tracking, providing insights into average response times and throughput.

These changes significantly enhance the usability and functionality of the metrics dashboard, providing a more robust framework for performance monitoring and data analysis.
2024-12-05 07:08:08 +08:00
c416c76790 feat(metrics): implement full metrics saving and enhance database schema
- Added functionality to save full metrics periodically and on application shutdown, improving data persistence.
- Introduced new database tables for performance metrics, status code history, popular paths, and referer history, enhancing data tracking capabilities.
- Updated the GetRecentMetrics function to utilize window functions for more accurate metrics retrieval and ensure non-negative values.
- Improved data handling by ensuring empty records are returned instead of null when no metrics are available.

These changes significantly enhance the metrics collection and storage process, providing a more robust framework for performance monitoring.
2024-12-05 06:47:39 +08:00
91d4686713 feat(metrics): add safe type conversion functions and enhance metrics data retrieval
- Introduced safe type conversion functions for memory usage, status code stats, top paths, and recent requests to prevent panics when accessing metrics data.
- Updated MetricsHandler to utilize these safe functions, improving the robustness of metrics handling.
- Refactored GetStats method in Collector to ensure all fields are initialized and to streamline the retrieval of metrics data.
- Enhanced the overall structure of metrics data for better clarity and reliability in reporting.
2024-12-03 18:17:25 +08:00
68c27b544b feat(metrics): enhance metrics functionality and configuration
- Added new dependency on github.com/mattn/go-sqlite3 for improved metrics storage.
- Updated main.go to initialize metrics collector with a new database path and configuration settings.
- Enhanced config.json to include additional metrics settings such as alert configurations and latency thresholds.
- Refactored internal metrics handling to support new metrics structures and improve data retrieval.
- Introduced a new metrics history endpoint for retrieving historical data, enhancing monitoring capabilities.
- Improved UI for metrics dashboard to include historical data visualization options.
2024-12-03 17:48:11 +08:00