Workspace folders

[paullizer]

(v0.238.024)

New Features

• Blob Metadata Tag Propagation

  • Document tags now propagate to Azure Blob Storage metadata when enhanced citations is enabled.
  • Automatic Sync: When tags are added, removed, or updated on a document, the corresponding blob's metadata is updated with a document_tags field containing a comma-separated list of tags.
  • Conditional: Only active when enable_enhanced_citations is enabled in admin settings; no blob metadata changes occur otherwise.
  • Cross-Workspace: Works for personal, group, and public workspace documents.
  • Non-Blocking: Blob metadata update failures are logged but do not prevent the primary tag propagation to AI Search chunks.
  • Files Modified: functions_documents.py.
  • (Ref: BLOB_METADATA_TAG_PROPAGATION.md)

• Enforce Workspace Scope Lock

  • New admin setting to control whether users can unlock workspace scope in chat conversations.
  • Enabled by Default: When enabled, workspace scope automatically locks after the first AI search and users cannot unlock it, preventing accidental cross-contamination between data sources.
  • Informational Modal: Users can still click the lock icon to view which workspaces are locked, but the "Unlock Scope" button is hidden and replaced with an informational message.
  • Backend Enforcement: Server-side validation rejects unlock API requests when the setting is enabled, providing defense-in-depth security.
  • Admin Toggle: Located in Admin Settings > Workspace tab in the new "Workspace Scope Lock" section.
  • Files Modified: config.py, functions_settings.py, route_frontend_admin_settings.py, admin_settings.html, chats.html, chat-documents.js, route_backend_conversations.py.
  • (Ref: ENFORCE_WORKSPACE_SCOPE_LOCK.md)

• Document Tag System

  • Comprehensive tag management system for organizing documents across personal, group, and public workspaces.
  • Tag Definitions: Tags with custom colors from a 10-color default palette (blue, green, amber, red, purple, pink, cyan, lime, orange, indigo) or user-specified hex codes. Colors assigned deterministically via character-sum hash.
  • Full CRUD API: 15 endpoints (5 per workspace type) for listing, creating, bulk tagging, renaming/recoloring, and deleting tags. Consistent API pattern across /api/documents/tags, /api/group_documents/<id>/tags, and /api/public_workspace_documents/<id>/tags.
  • Bulk Tag Operations: Apply, remove, or replace tags on multiple documents in a single operation with per-document success/error reporting.
  • AI Search Integration: Tags propagate to all document chunks via propagate_tags_to_chunks(), enabling OData tag filtering during hybrid search with AND logic (document_tags/any(t: t eq 'tag')).
  • Tag Validation: Max 50 characters, alphanumeric plus hyphens/underscores only, normalized to lowercase, duplicates silently deduplicated.
  • Tag Storage: Personal tags in user settings, group tags on group Cosmos document, public workspace tags on workspace Cosmos document.
  • Files Modified: functions_documents.py, functions_search.py, route_backend_documents.py, route_backend_group_documents.py, route_backend_public_documents.py.
  • Files Added: static/json/ai_search-index-user.json, static/json/ai_search-index-group.json, static/json/ai_search-index-public.json.
  • (Ref: Document Tag System, AI Search OData filtering, cross-workspace tags, DOCUMENT_TAG_SYSTEM.md)

• Workspace Folder View (Grid View)

  • Toggle between traditional list view and folder-based grid view for workspace documents via radio buttons.
  • Tag Folders: Color-coded folder cards displaying tag name, document count, folder icon, and context menu (rename, recolor, delete).
  • Special Folders: "Untagged" folder for documents with no tags and "Unclassified" folder for documents without classification (when classification is enabled).
  • Folder Drill-Down: Click a folder to view its contents with breadcrumb navigation, in-folder search, configurable page sizes (10, 20, 50), and sort by filename or title.
  • Grid Sort Controls: Sort folder overview by name or file count with ascending/descending toggle.
  • View Persistence: Selected view preference saved to localStorage and restored on page load.
  • Tag Management Modal: Step-through workflow for creating, editing, renaming, recoloring, and deleting tags with color picker.
  • Cross-Workspace Support: Equivalent grid view and tag management available in group workspaces (inline JS) and public workspaces.
  • Files Added: workspace-tags.js (1257 lines), workspace-tag-management.js (732 lines).
  • Files Modified: workspace.html, group_workspaces.html, public_workspaces.html, public_workspace.js.
  • (Ref: Folder view, tag management modal, grid rendering, WORKSPACE_FOLDER_VIEW.md)

• Multi-Workspace Scope Management

  • Select from Personal, multiple Group, and multiple Public workspaces simultaneously in the chat interface.
  • Hierarchical Scope Dropdown: Organized sections with checkbox multi-selection and "Select All / Clear All" toggle with indeterminate state support.
  • Scope Locking: Per-conversation lock that freezes workspace selection after the first AI Search. Three-state machine: null (auto-lockable) → true (locked) → false (user-unlocked) → true (re-lockable).
  • Lock Indicator: Visual lock icon with tooltip showing locked workspace names. Locked workspaces appear grayed out in the dropdown.
  • Lock/Unlock Modal: Dialog for manually toggling scope lock per conversation.
  • Lock Persistence: Lock state stored in conversation metadata via PATCH /api/conversations/<id>/scope_lock.
  • Workspace Search Container: Multi-column flex layout (Scope → Tags → Documents) with connected card UI and viewport boundary detection.
  • Files Modified: chat-documents.js, chat-messages.js, chats.html, route_backend_chats.py, route_backend_conversations.py.
  • (Ref: Multi-workspace selection, scope locking, search container layout, MULTI_WORKSPACE_SCOPE_MANAGEMENT.md)

• Chat Document and Tag Filtering

  • Checkbox-based multi-document selection replacing the legacy single-document dropdown in the chat interface.
  • Custom Document Dropdown: Checkboxes for each document with real-time search, "All Documents" option, and selected count display ("3 Documents").
  • Scope Indicators: Each document labeled with its source workspace: [Personal], [Group: Name], or [Public: Name].
  • Multi-Tag Filtering: Checkbox dropdown for selecting tags to filter the document list. Classification categories shown with color coding when enabled.
  • Dynamic Tag Loading: Tags load and merge across all selected scope workspaces with aggregated counts.
  • DOM-Based Filtering: Non-matching documents removed from the DOM (not hidden via CSS), following project conventions. Removed items stored for restoration when filters change.
  • Backend Integration: Selected document IDs and tags sent in chat request body. Backend constructs OData AND filter: document_tags/any(t: t eq 'tag1') and document_tags/any(t: t eq 'tag2').
  • Files Modified: chat-documents.js, chat-messages.js, functions_search.py, route_backend_chats.py, chats.html.
  • (Ref: Multi-document selection, tag filtering, OData search integration, CHAT_DOCUMENT_AND_TAG_FILTERING.md)

Bug Fixes

• Citation Parsing Bug Fix
  • Fixed citation parsing edge cases where page range references (e.g., "Pages: 1-5") failed to generate correct clickable links when not all pages had explicit reference IDs in the bracketed citation section of the AI response.
  • Root Cause: The parseCitations() function only generated links for pages with existing [doc_prefix_N] bracket references, leaving pages without explicit references as non-functional text.
  • Solution: Added auto-fill logic using getDocPrefix() to extract the document ID prefix from known reference patterns and construct missing page references (e.g., if [doc_abc_1] exists, infer doc_abc_2 through doc_abc_5).
  • Files Modified: chat-citations.js.
  • (Ref: Citation parsing, page range handling, CITATION_IMPROVEMENTS.md)

User Interface Enhancements

• Extended Document Dropdown Width

  • Widened the document selection dropdown in the chat interface for improved readability of long filenames. Dropdown width now dynamically adapts to the parent container.
  • Files Modified: chat-documents.js.
  • (Ref: Document dropdown, UI readability)

• Enhanced Citation Links

  • Robust inline citation links with support for both inline source references and hybrid citation buttons.
  • Metadata citation support for viewing extracted document metadata including OCR text, vision analysis, and detected objects via the enhanced citation modal.
  • Improved error handling in citation JSON parsing with graceful fallback for malformed citation strings.
  • Files Modified: chat-citations.js, chat-enhanced-citations.js.
  • (Ref: Citation rendering, metadata citations, enhanced citation modal, CITATION_IMPROVEMENTS.md)

[Copilot]

Pull request overview

This PR introduces a comprehensive workspace folders feature set for organizing and managing documents across personal, group, and public workspaces. The implementation includes four major features: a Document Tag System with full CRUD operations, a grid-based Folder View with drill-down navigation, Multi-Workspace Scope Management with automatic locking, and enhanced Chat Document/Tag Filtering. The PR also includes citation parsing improvements and automatic AI Search index field fixes.

Changes:

• Document Tag System: 15 API endpoints (5 per workspace type) for tag management with color-coded UI and AI Search integration
• Workspace Folder View: Toggle between list/grid views with tag-based folders, breadcrumb navigation, and sortable contents
• Multi-Workspace Scope Management: Hierarchical checkbox dropdown for selecting multiple workspaces with per-conversation scope locking
• Chat Document/Tag Filtering: Multi-select dropdowns for documents and tags with DOM-based filtering
• Citation improvements and auto-fix for Azure AI Search index fields

Reviewed changes

Copilot reviewed 43 out of 45 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
config.py	Version bump to 0.238.024
docs/explanation/release_notes.md	Added comprehensive release notes for v0.238.024
docs/explanation/features/v0.238.024/*.md	Four new feature documentation files
functions_documents.py	Added tag management functions (normalize, validate, get_workspace_tags, propagate_tags_to_chunks)
functions_search.py	Added build_tags_filter() and multi-group/multi-document support
route_backend_*.py	Added tag CRUD endpoints across all workspace types
route_backend_conversations.py	Added scope lock PATCH endpoint
route_backend_chats.py	Integrated tags_filter and active_group_ids parameters
functions_conversation_metadata.py	Implemented scope lock logic
utils_cache.py	Updated cache keys for multi-group and tags support
templates/*.html	Added grid view UI, scope dropdown, tag management modals
static/js/workspace/*.js	New modules for tag management and grid rendering
static/js/chat/*.js	Updated for multi-workspace scope and tag filtering
static/json/ai_search-index-*.json	Added document_tags field to all three index schemas
route_backend_settings.py	Added auto-fix functionality for missing index fields
CLAUDE.md	New project instructions document

[Copilot]

Pull request overview

Copilot reviewed 48 out of 50 changed files in this pull request and generated 3 comments.

[Copilot]

The auto_fix_index_fields function lacks proper error handling for several potential failure modes: JSON file read errors, invalid schema format, and Azure AI Search API failures. Consider adding specific try-except blocks around file operations and index operations with appropriate error messages and logging using log_event from functions_appinsights.py as per coding guidelines.

Suggested change

	return {'error': str(e)}
	# Log unexpected errors for observability and debugging
	error_message = f"auto_fix_index_fields failed for idx_type={idx_type}: {str(e)}"
	try:
	log_event(
	"auto_fix_index_fields_error",
	{
	"error": str(e),
	"index_type": idx_type,
	"user_id": user_id,
	"admin_email": admin_email or "",
	},
	)
	except Exception:
	# Avoid raising from logging failures; keep original exception context
	pass
	return {'status': 'error', 'error': error_message}

[Copilot]

The version mismatch between ENFORCE_WORKSPACE_SCOPE_LOCK.md (line 4) which states "Version: v0.238.025" and the actual version in config.py (line 91) which is "0.238.024" creates inconsistency. The documentation should match the actual release version.

[Copilot]

The version mismatch between BLOB_METADATA_TAG_PROPAGATION.md (line 4) which states "Version: v0.238.025" and the actual version in config.py (line 91) which is "0.238.024" creates inconsistency. The documentation should match the actual release version.