Data Imports
- Amanda Thomas
- Keith Mangold
Data Import Schedules
This article documents the procedures required to import data into Aeros LIVE using 1) spreadsheets provided by Cultura and 2) data or files retrieved from third party software systems.
Get the Data Import Templates here
Steps to create a Data Import
To create a data import, the following steps must be completed:
Set up a Data Import Schedule. This is where an import type is initiated. Directories, Scheduling, and History is set up in the Data Import Schedule.
Set up a Data Import Map. In the data import map, files set up in the Data Import Schedule are mapped to matching Aeros LIVE values. This process is different for each type of import.
Run the Data Import Schedule to Process the files.
The following pages explain these steps in detail.
Types of Data Imports
Spreadsheets provided by Cultura for direct import. Any description in the Import Type drop down that starts with the word “Aeros LIVE” indicates an import via spreadsheet.
Data or files retrieved from third party software systems. Any description in the Import Type drop down that does not start with the word “Aeros LIVE” indicates a third party spreadsheet
Currently, Aeros LIVE has the capability to import the following:
Steps to Create a Data Import Schedule
Navigate to the Data Import Schedule through System -> Data Imports -> Data Import Schedules as shown below:
After selecting Data Import Schedules from the System Menu, you will see the following screen:
Import Details
Import Type: Select from the list of applicable types of data that can be brought into Aeros LIVE. This is a required field.
Code: The corresponding Import Code. This is a required field and must be unique.
Description: The description of the import.
Source Folder: Will contain the file to be processed whether its manually inserted into the folder or has been pushed from a third party software service such as AEI, PMSI, WEM, or Repete.
Output Folder: After an import is processed, the file will be renamed to add a prefix detailing the date the file was moved. The original file name remains in the last part of the processed file name. To reprocess the file, move the file back to the Source Folder and rename it to its original name.
Overwrite duplicate records: Overwrite duplicate records by selecting the checkbox. If all data in a record matches, the record is overwritten when this box is checked. If the box is not checked it will still not duplicate the data but create a message that the record is a duplicate and belongs to another batch.
To utilize the scheduler for imports you will need to install our scheduler for LIVE. Please contact customer service to get this installed.
Schedule Imports: A schedule can be selected that will allow files that are inserted into the source folder to be automatically ran at a selected time (hourly, daily, weekly and monthly).
Schedule Time: The time the import will run.
Data Map Tab
The second tab of the schedule configuration “Data Map” lists, for each field to be imported, the name of the Source Field and the name of the Destination Field within Aeros LIVE. This is a read-only screen for reference.
The spreadsheet for each import can be downloaded from the "Data Map" tab of the data import schedule form by clicking the button in the top right corner.
Import History Tab
The Import History tab shows a complete history of every file processed through the selected Import Schedule that have yet to be archived.
For each file processed, a record will display when you first click on the Import History Tab
Start Time. Date and time the import process began.
End Time. Date and time the import process ended.
Batch Number. Aeros LIVE system identifier for the batch number
Run Type. Manual or Automatic – indicates how the import was processed.
File Path. The source file original path, before the file was processed
Records in File. Number of records found in source file.
Records Inserted. Number of records inserted into Aeros LIVE. You will note that this number does not generally match the records in file. This is because a source file may contain data that will be inserted in several Aeros LIVE forms.
For example, if a source file contains 8 records, and those records will be inserted into the Mortality, Ending Feed, Egg Production, and Recordkeeping forms, you should expect to see 8x4 records inserted, a total of 32. That is because 8 records are being inserted into 4 Aeros LIVE forms each.
Errors. Indicates how many records contained errors and were not processed. Error details can be seen when you select to View this Batch.
Warnings. If the system found issues that may be problematic, but did not break validation rules, a Warning message will be created. Warning details can be seen when you select to View this Batch.
Messages. System generated messages are created with each import. At least 3 are expected:
Batch process begins
File is moved to the completion path
Batch process is complete
View This Batch. The “View this Batch” button in the upper right hand corner contains a view the details of any import batch when the record is selected in the grid. From here, view Errors, Warnings, and Messages. This can also be accessed if you double click any row in the data grid.
Rollback Selected Batch. Delete the selected batch and any imported records. To completely delete the imported data for any imported file, click this button and the data will be removed.
Archive Batch. Once you have confirmed a batch, you can remove it from the list to prevent it from ever being deleted or reprocessed. This button will remove the import from the Import History tab. This will prevent an accidental reprocessing or deletion of a batch. Use this when you are certain data within a file never needs to be reprocessed and you want to protect the data from being removed.
Password Verification
We added password approval to be able to archive the import batch as well as delete an import configuration to ensure this is what you really want to do. Once you delete an import configuration any processed batches will be removed from the application and can only be recovered from the database. The password is the same as your login password.
Note: If you accidentally remove a batch that you need, Aeros can recover it. Contact our Customer Support for assistance. rcm.customersupport@culturatech.com
In the history above the import processed the example spreadsheet but out of the 252 records in the file none of the records were inserted. By double clicking on the row, right click view, or selecting the row and then the green button on the upper right corner of the form, the user can view a batch log that will show what went wrong.
Batch Log Tab.
This tab displays the details of every Error, Warning, and Message generated during the Import Schedule Run for a file. The Source ID/Key field will display either the original file name, or the record within the file, depending on the message.
Imported Rows.
This tab will display the details of each imported record: What group the record applied to, which database table the record was stored in, and the actual values. The source file record number is also displayed.Finally, another feature of the import data process is the functionality to roll back records that were imported into the database. By selecting a batch and using the delete button on the upper right hand corner of the form the records will be removed.
Import History Options
The following functions are available within the Import History tab. Highlight any row in history and click a button to perform the following processes.
Note: If a file located in the directory should not be processed, you should expect it to show up in error on the history screen. This verifies that the system found the file and rejected it, because the file does not contain the appropriate data for processing. Files found in error are not processed, are not moved, and are not renamed.
Currently, Aeros LIVE has the capability to bring in the following pieces of Data:
Flocks - Quick Flock | Feed Delivery | Culls | Egg Inventory |
Estimated Placements | Ending Feed Inventory | Egg Production | Record Keeping |
Estimated Depopulations | Mortality | Egg Shipment | System Tables |
Condemnations | Expenses | Sales/Live Hauls/Depops |
Data Import Maps
Data Import Schedule gives the user the ability to import data into Aeros LIVE using third party software files.For third party applications such as AEI, Aeros LIVE has the ability to map source house codes that differ from the house codes utilized in Aeros LIVE. The Data Import Maps allows for a house code on the source file to be mapped to the house code in Aeros LIVE.UsageData Import Maps are used for importing data from third party software vendors only. Typically, Cultura will configure these maps for you during implementation.
Navigate to the Data Import Schedule through System -> Data Imports -> Data Import Schedules as shown below:
Instructions: After selecting Data Import Maps from the System Menu, you will see the following screen:
Import Type. Select an import type to view the mappings for any of the following systems
Each type of system will have a different type of mapping. Once an import type is selected, if records are mapped, they will display in the grid.
House Code. The first column will reference the destination house in Aeros LIVE receiving the data. Aeros LIVE dynamically assigns records to the group or flock placed in a house based on the date of the incoming record. The display will show the Company Code (dash) Division Code (dash) Farm Code (dash) House Code.
Import Schedule. The Import Schedule column references the Data Import Schedule Code created in the first section.
Source Key. The third column will reference the source key from the third party system.RestrictionsThese values can be edited as needed for changes and additions to the 3rd party software or the Aeros LIVE Facility Management structure. Only one House Code may be mapped to each Source Key, no duplicates are allowed.
Summary and ReferencesThis document should assist anyone utilizing the Aeros LIVE Data Imports module. It is a general overview of the module and how it works.
During implementation, you will be provided with a document specific to your organization, as each data import is likely to be customized per client.
If not provided, please contact Cultura Customer Support and specify the system for which you require documentation.
Types of Data Aeros LIVE Data Imports will Not Import
If any data requires manipulation prior to import, it will not be processed through this module. Any data requiring the process of capturing, reviewing, and editing data before it is processed in Aeros LIVE will be handled through our Cultura Application Interface system.
Download the data Import templates here.
The spreadsheet for each import can be downloaded from the "Data Map" tab of the data import schedule form by clicking the button in the top right corner.