Posts

Showing posts from May, 2018

Error ‘Archive is corrupted’ While Unzipping through Pentaho

Image
Error ‘Archive is corrupted’ While Unzipping through Pentaho Occasionally, this issue occurs when you are trying to unzip the archive file you received through FTP transfer. Resolution for this is two-step process. Step 1: Checking the FTP transfer data type of file. Please follow the below steps:        Open your FileZilla FTP client.     Click on ‘Edit’ tab and select ‘Settings’. This will open pop up.   Under ‘Transfers’ section, select ‘File Types’. Please choose ‘Binary’ as your Default transfer type. Note: Setting the Binary as your default transfer type will prevent your files to become corrupted while ftp transit. Step 2: Binary Mode check box under ‘Get a File with FTP’ component before ‘Unzip file’ component in Pentaho should be marked as check. Once you tick the Binary Mode under ‘Get a File with FTP’ component, it will download the zip file in Binary format preventing your archive files to become corrupted. Please refer below snip

Data Archival using Automated Sliding Window Partitioning - Part I

Image
Introduction One of the most common requirement in day-to-day production environment maintenance is data archival. As, the amount of data grows due to constant inflow of data. The overall performance of the application might also go down due to the ever increasing number of records in the underlying database tables. This is where data archival comes into picture. Data archival simply means to move the data from the table which is being used by any downstream application to an archival table. Ofcourse, it needs to be ensured that the data archival process is well thought off in terms of its impact on the data available for consumption for downstream applications. It means for example, we do not want a scenario wherein a report was generated on the yearly sales data and we archived 6 months worth of sales data, thus ending up showing only 6 months sales in an yearly sales report. For this post, we will focus on the underlying concept of the 'sliding window partitioning&#