VFBquery v0.5.1

VFBquery v0.5.1 Release Notes

Overview

VFBquery v0.5.1 is a maintenance release that builds upon the major performance and feature enhancements introduced in v0.5.0. This release focuses on refining the caching system, improving test coverage, enhancing documentation, fixing minor bugs, and streamlining CI/CD workflows. These improvements ensure better reliability, performance, and developer experience for VirtualFlyBrain data exploration.

🚀 Major Features

Enhanced Caching System

• Background Caching: Implemented background caching for expensive queries in SolrResultCache to improve responsiveness
• Advanced Result Handling: Enhanced cached result handling to support slicing for dict types with 'rows' and 'queries' keys
• Force Refresh Option: Added force refresh capability to caching functions for bypassing cache when needed
• Comprehensive SOLR Caching: Extended SOLR-based caching to similarity queries, term info retrieval, and neuron-related functions
• Unified Caching Architecture: Refactored caching implementation to use SOLR-based caching throughout the codebase for better performance and simplified management

🔧 Improvements

Testing & Quality Assurance

• Expanded Test Coverage: Added initial test cases for VFB term and instance retrieval functions
• Improved Test Logic: Refactored neuron part tests to set minimum expected counts and enhance cache key handling for DataFrame results
• SOLR Availability Handling: Added helper methods to skip tests when SOLR server is unavailable
• Test Result Management: Refactored test results update to use JSON format for better data handling and fixed update logic for appending new templates

Documentation & Examples

• Enhanced README: Added detailed metadata and example queries for JRC2018Unisex brain template, refactored medulla examples, and improved overall clarity
• Consistent Formatting: Refactored README thumbnails for consistent escaping and added sorting functions for data rows in test examples
• Code Quality: Updated test scripts and README for improved functionality and maintainability

Code Quality & Maintenance

• Import Refactoring: Updated import statements to use relative paths and improved code structure readability
• Parser Enhancements: Refactored readme_parser to remove regex for boolean conversion and added update_readme script for JSON block replacement
• Code Block Processing: Added logic to skip processing Python code blocks containing import statements

🐛 Bug Fixes

• Debug Cleanup: Removed debug print statements from cached functions and Owlery client
• URL Corrections: Fixed license icon URL to use HTTP and updated term info serialization test URLs to HTTPS
• Workflow Fixes: Restored original print statements for SOLR unavailability in example execution
• Query Consistency: Refactored neuron region connectivity queries to use 'region' instead of 'label' for clarity in preview columns

🔄 CI/CD Enhancements

• SOLR Integration: Added SOLR availability checks to GitHub Actions workflows with proper exception handling
• Workflow Optimization: Reordered dependency installation steps and removed duplicate conditions in examples workflow
• Test Result Updates: Added automated steps to update performance test results in CI/CD pipeline
• Path Corrections: Fixed paths for updating test results in GitHub Actions workflow

📊 Performance Metrics

• Caching Efficiency: Improved cache hit rates through enhanced SOLR-based caching implementation
• Background Processing: Reduced blocking operations with background caching for expensive queries
• Test Reliability: Better handling of external service dependencies (SOLR) in automated testing

📦 Dependencies

• No dependency changes in this release
• Enhanced compatibility with existing VFB infrastructure through improved error handling

🙏 Acknowledgments

This release includes contributions from the VirtualFlyBrain development team, focusing on user feedback regarding performance, reliability, and usability. Special thanks to automated testing and CI/CD improvements that ensure consistent quality.

---

Installation:

pip install vfbquery==0.5.1

Full Changelog: v0.5.0...v0.5.1

v0.5.0

VFBquery v0.5.0 Release Notes

Overview

VFBquery v0.5.0 introduces significant performance improvements, new connectivity queries, enhanced caching capabilities, and comprehensive test coverage. This release focuses on making VFBquery more robust, faster, and feature-complete for VirtualFlyBrain data exploration.

🚀 Major Features

New Connectivity Queries

• Neuron-Neuron Connectivity: Query neurons connected to a given neuron with synaptic weight information
• Neuron-Region Connectivity: Find brain regions where neurons have synaptic terminals
• Enhanced OWL-based Queries: Improved implementation of lineage clones, fasciculating neurons, and innervating tracts/nerves

Performance Enhancements

• Parallel Processing: Concurrent query execution in fill_query_results for faster term info retrieval
• Extended Timeouts: Increased HTTP request timeouts from 20 to 40 minutes for complex OWL reasoning queries
• SOLR Timeout Increase: Extended SOLR request timeout from 30 to 120 seconds
• Optimized Caching: Smart cache extraction for limited result queries from full cached datasets

Advanced Caching System

• Multi-layer Caching: Combined SOLR and memory caching for optimal performance
• Cache Validation: Enhanced validation to prevent caching of error results
• Preview Parameter Support: Intelligent caching based on preview requirements
• Fallback Mechanisms: Graceful degradation when Neo4j is unavailable

🔧 Improvements

Query Enhancements

• Standardized field naming across all connectivity queries (using label consistently)
• Improved Cypher query patterns for better performance
• Enhanced error handling and retry logic for connection issues
• Better IRI handling and URL construction for OWL queries

Testing & Quality

• Comprehensive test suites for all major query types
• Performance benchmarking with configurable thresholds
• Enhanced CI/CD workflows with proper environment handling
• Mocked testing support for development environments

Documentation

• Updated README with detailed examples and improved formatting
• Comprehensive VFB Queries Reference documentation
• Enhanced API documentation and usage examples
• Performance test reporting and analysis

🐛 Bug Fixes

• Fixed caching issues with error results
• Improved error handling in Owlery client
• Corrected query parameter handling and validation
• Enhanced debug logging for troubleshooting

📊 Performance Metrics

• Significant reduction in query execution times through parallel processing
• Improved cache hit rates with enhanced validation
• Better handling of slow queries with extended timeouts
• Optimized memory usage in caching system

🔄 Migration Notes

• Breaking Change: Connectivity queries now use label field consistently instead of descriptive field names
• Timeout Changes: Default timeouts have increased - update client code if you have custom timeout handling
• Cache Behavior: Enhanced caching may change performance characteristics for repeated queries

📦 Dependencies

• Enhanced compatibility with latest VFB infrastructure
• Improved error handling for network connectivity issues
• Better integration with VFB Connect services

🙏 Acknowledgments

This release includes contributions from the VirtualFlyBrain team and community feedback. Special thanks to users who provided performance feedback and bug reports that helped shape these improvements.

---

Installation:

pip install vfbquery==0.5.0

Full Changelog: v0.4.1...v0.5.0

v0.4.1

What's Changed

• Adding initialize_vfb_connect by @Robbie1977 in #30
• Fix for bracket encoding in label by @Robbie1977 in #31
• fix for encoding in thumbnails by @Robbie1977 in #32
• Fix for testing and encoding by @Robbie1977 in #33

Full Changelog: v0.4.0...v0.4.1

v0.4.0

What's Changed

• Cache Improvements by @Robbie1977 in #29
  This implements comprehensive caching improvements for VFBquery to address cold start performance issues and reduce response times from minutes to seconds. The changes introduce multiple layers of caching, including in-memory, disk-based, and SOLR-based caching systems, all enabled by default with production-ready configurations.

Key changes:

Added SOLR-based result caching with 3-month TTL for server-side optimization
Implemented multi-layer memory and disk caching with intelligent LRU eviction
Enhanced get_instances function with fallback SOLR implementation when Neo4j is unavailable
Added comprehensive performance testing infrastructure and monitoring

Full Changelog: v0.3.4...v0.4.0

Current Beta Release (2025-09-03)

Fixes for handling of non-serialisable data

Full Changelog: v0.3.3...v0.3.4

Current Beta Release (2025-06-16)

What's Changed

• Expanded output by @Robbie1977 in #27
• fix: update test case to use correct term info ID in serialization cl… by @Robbie1977 in #28

Full Changelog: v0.3.2...v0.3.3

Current Beta Release (2025-06-15)

What's Changed

• Expanded output and test fixes by @Robbie1977 in #25
• moving to cached calls direct from solr by @Robbie1977 in #26

Full Changelog: v0.2.12...v0.3.2

Current Beta Release (2025-03-12)

What's Changed

• moving to cached calls direct from solr by @Robbie1977 in #26

Full Changelog: v0.3.0...v0.3.1

Current Beta Release (2025-03-11)

What's Changed

• Expanded output and test fixes by @Robbie1977 in #25

Full Changelog: v0.2.13.1...v0.3.0

Current Beta Release (2024-04-09)

encoding labels inside markdown links
Full Changelog: v0.2.12...v0.2.13