DLI now supports Apache Hudi 0.15 for Spark workloads.
Data Lake Insight (DLI) – new functions
New features
1. Spark Hudi upgraded to version 0.15
Enhancements
- Spark SQL General Hudi has been upgraded to version 0.15.
- Independent image building is now supported.
- DLI Hudi lock mechanisms have been adapted for compatibility with Hudi 0.15.
Benefits
- Access to the latest Hudi improvements and enhancements.
- Improved compatibility for modern data lake workloads.
- More flexible deployment and image management processes.
2. Secondary confirmation for Critical Management Parameters
A new protection mechanism has been introduced for critical platform configuration changes.
What's new
A secondary confirmation process is now required when modifying sensitive parameters such as: luxor.metaServer.upgradeMode.enable
(This parameter affects all Redis instances on the DLI management plane.)
Benefits
- Reduces the risk of accidental configuration changes.
- Improves operational safety and governance.
- Helps protect platform stability during maintenance activities.
3. Extended SQL job idle status retention
DLI now supports a longer retention period for SQL job idle status information.
What's new
- The maximum configurable retention period has been increased to seven days.
- Backend support has been enhanced to manage extended retention periods.
Benefits
- Improved visibility into historical job activity.
- Better support for troubleshooting and operational analysis.
- Increased flexibility for monitoring and compliance requirements.
Weitere Informationen finden Sie im Bereich des Data Lake insight (DLI) Help Center.
Do you have questions?
We answer your questions about testing, booking and use – free of charge and individually. Try it!
Hotline: 24 hours a day, 7 days a week
0800 3304477 from Germany / 00800 33044770 from abroad