Release Notes

Note: SDK v1.0 is compatible with Amorphic v3.3 and above

Features (19)

  1. AI Projects & Threads – Full lifecycle management for AI Projects has been introduced, enabling users to maintain multi-turn conversational threads, manage context, and securely organize AI interactions within a centralized workspace.

  2. Dataset AI Metadata – Automated workflows have been added to generate, review, and approve AI-generated labels and summaries, significantly enhancing dataset discoverability and context.

  3. NL2SQL Enhancements – Users can now upload and manage specific training documents and trigger synchronization jobs to improve the accuracy and context of the natural language-to-SQL chatbot.

  4. Model Management & Configurations – Expanded capabilities allow administrators to govern AI models globally, manage model access, update invocation profiles, and seamlessly migrate RAG engines.

  5. Machine Learning Models – Comprehensive capabilities have been included to track, register, and manage standard machine learning models alongside generative AI assets.

  6. ETL Job Management – A robust suite for managing ETL jobs has been introduced, providing users with the ability to track executions, manage scripts, utilize job bookmarking, and trigger automated error diagnosis.

  7. Shared Job Libraries – Developers can now create, synchronize, and update shared code libraries across multiple ETL jobs, promoting code reuse and organizational standardization.

  8. Data Pipelines – Advanced orchestration capabilities for data pipelines have been added, complete with granular node-level logging and detailed file load manifests for better observability.

  9. Iceberg Optimizers – Users can now configure and monitor automated optimization routines—such as compaction, snapshot retention, and orphan file deletion—to maintain the performance and storage efficiency of Iceberg datasets.

  10. Business Glossaries – New governance tools have been introduced to create, manage, and enforce business glossaries and individual terminology across the organization.

  11. Enhanced Catalog Discovery – Catalog search capabilities have been upgraded, enabling users to perform deep searches, map complex entity relationships, and generate custom actions for discovered assets.

  12. Datasource Asset Collaboration – Users can now seamlessly track, update, and collaborate on datasource assets through integrated commenting and synchronization workflows.

  13. Data Lineage V2 – Lineage tracking has been upgraded to provide high-resolution visibility into resource dependencies and specific historical lineage events.

  14. Advanced Query Execution – Fine-grained control has been added to safely start, stop, and monitor SQL queries, alongside secure methods for downloading query results.

  15. Query Workbooks – Dedicated workspaces have been introduced where users can create, organize, and execute complex queries in an isolated and structured environment.

  16. Athena Integration – Streamlined secure data access is provided by direct retrieval of IAM credentials for Amazon Athena sessions.

  17. Datalab Management – Extensive control over Datalab environments has been delivered, supporting automated lifecycle configurations, active session tracking, and seamless integration with applications like JupyterLab and RStudio.

  18. Centralized Parameter & Code Management – Dedicated components have been added for managing integrated code repositories and a unified parameter store to streamline cross-platform configuration management.

  19. Scheduling & Event Rules – Orchestration capabilities have been expanded, allowing users to define complex event rules and comprehensively manage the lifecycle and execution of scheduled jobs.

Enhancements & API Payload Schema Changes (10)

This release introduces significant enhancements to existing platform capabilities, primarily driven by expanded payload configurations and query parameters that provide deeper control over resources:

AI & Machine Learning Enhancements

  • Agent Cost Tracking & Parameters – AI Agents can now be configured with CostTags to track associated AWS Bedrock expenses. Additionally, agents now support ResourceAccessMetadata.ParameterAccess, allowing them to securely interact with the parameter store.

  • Rich AI Notes – AI Notes have been upgraded to support full NoteContent, explicit ProjectId association for better workspace organization, and the ability to track referencing Sources.

  • Advanced Knowledge Base Configuration – When creating Knowledge Bases, users can now explicitly define the DataStoreType (e.g., OpenSearch Serverless vs. RDS), KnowledgebaseType, and TenantName to better support multi-tenant architectures and specific retrieval requirements.

  • AI Model Visibility & Controls – Model listing endpoints now support robust pagination (limit, offset) and sorting. Furthermore, model configurations have been expanded to accept CostTags, InferenceType for regional routing, and specific ModelParameters.

Data Management & Governance Enhancements

  • Expanded Dataset Configurations – Dataset creation and update workflows have been heavily enhanced. Users can now define DataCategory and DataCategoryConfig, establish IdentityAttributeValues, and toggle advanced integrations like IsAutoMLEnabled, IsBDAExtractionEnabled, and IsGovernedTable.

  • Tag-Based Access Control (TBAC) – Data permission listings now support an is_tbac flag, providing better visibility into permissions granted via tag-based access policies rather than direct user/group assignments.

  • User-Friendly Data Classifications – Data classification metadata now supports a customizable DisplayName, improving readability across the platform interface.

File Handling & System Enhancements

  • Partition-Aware File Uploads – When generating S3 presigned URLs for standard or sample file uploads, users can now pass PartitionKeys to ensure data is correctly routed and partitioned upon ingestion.

  • Targeted File Deletion – Dataset file deletion capabilities have been enhanced to accept FileSearchTags, allowing for bulk deletion based on specific metadata rather than just file paths.

  • Role Management Upgrades – Access role management has been improved with support for pagination/sorting, and the ability to explicitly define RoleManagers during role updates.

Deprecated Features & Removed APIs (4)

To streamline the platform and transition to more robust underlying systems, the following legacy capabilities and their associated API endpoints have been removed in v1.0.0. Please migrate to the recommended alternatives:

  • Legacy Search Systems – The legacy resource and dataset search endpoints (search_resources, search_datasets) have been completely deprecated. Migration: All platform search functionality has been centralized; users should now utilize the new unified Catalog Search APIs.

  • Legacy Dataset Lineage – The older v1 dataset lineage endpoints (get_dataset_lineage, create_or_update_dataset_lineage) have been removed. Migration: Transition all lineage tracking to the new, highly detailed Lineage V2 capabilities.

  • Standalone Guard Rail Configurations – The endpoints used to configure component-specific guard rails (configure_component_guard_rails, delete_component_guard_rail_configuration, list_component_guard_rail_configurations) have been removed. Migration: Guard rail enablement and model selection have been consolidated into the unified AI Configurations API (update_ai_configurations).

  • AI Chat File Upload URL – The legacy endpoint for generating AI chat file upload URLs (create_ai_chat_file_upload_url) has been deprecated in favor of the modernized AI Projects and Threads architecture.