Calem Blogs

Blogs of CalemEAM

Bulk Update by Data Export and Upload

 The integration module allows administrators to perform bulk data initialization or updates.

  • You can craft an excel template from a report and export data to store off the excel file.
  • Use the same excel file to populate data and upload data to Calem.

0. Best Practice for Data Upload
Data upload may insert or update your data. It is critical that you follow the rules here to avoid updating your data unexpectedly.

Rule 1. Upload in Sandbox First
Before updating your production data you should perform the upload in Sandbox first. Here're the detailed flow.
  • The sandbox is synced from production for upload test
  • When the upload is ready for production, you may optionally sync again from production to sandbox for comparison after upload to production.

 Rule 2. Data Upload is UPSERT

Data Upload is an operation of upsert, i.e, a row is inserted into a table first. If there is no key violation a new record is created. Otherwise, an existing record is updated.

1. Summary

Calem allows one to upload data in excel files.

  • Menu: Integration | Data Upload
  • Excel template files for tables are provided by Calem. For instance, the blog Data Upload Part 1: Inventory Setup includes excel template files for inventory upload.
  • Alternatively, a Calem report may be exported to Excel, updated, and upload back into Calem for data updates.

2. Data Upload

In Calem release R10 you can batch update data through data export and import. This is recommended approach to update multiple data rows.

Here are the steps:

  • To update data the key correct field or fields must be included for upload to be successful.
    • Use "Customize" button on a list report to turn a report into design mode. Right click on the list caption row to add or remove columns you need to upload to the list. So, the final list view should include all the fields you need to populate data with.
    • See this blog for report list customization.
  • Use a report list to export data to be updated. For instance, "Asset module | Report | Asset report list". 
    • Narrow down the list through search before exporting the list.
    • If the report list does not include the fields you need to change or add. Customize the report list to include the fields you need (see this blog ). 
  • Edit the excel file and save changes. 
    • You may narrow down the columns to upload from your excel file.
    • For instance, the excel includes Asset Tag (required to update an asset), class, and other fields in the screenshot below. If you only need to update class fields, the excel file can include only Asset Tag and class in the second excel screenshot below.
  • Upload the edited excel file to the database. The menu path is "Integration module | Data Upload"

3. Null Value

A null value means no value in a database field. One can set a database field to null in Calem by "_null_" (underscore, "null", underscore). 

  • A lookup field can be set to null (such as the parent Id of an asset).
  • A string field can be set to null (such as the description of an item)
  • A numerical value can be set to null (such as an asset's weight).
  • A datetime field can be set to null (such as an asset's start date).

4. Id Column

Each record in Calem has an "Id" field. The "Id" field uniquely identify a data row. You may include the "Id" field in your export if you need to modify the key fields of a row. For instance, if you have to change item numbering or asset tags, you need to include the "Id" field in the export so Calem knows which row to apply the key field changes such as item number and asset tag.

 5. Data Validation Configuration

By default Calem Enterprise Upload utility will verify lookup values and will fail the upload if a lookup value is invalid. For instance, a location of an asset is not found in Calem when uploading an asset with a reference to that location. Calem will fail the upload.
Alternatively, you may turn off the data validation so that Calem will create missing lookup values. If the creation is successful the upload will go through successfully.
Here is the recommendation about the data validation configuration:

  • The data validation should be left enabled by default all the time. This is important so data would not be added to your service by accident. For instance, a location "Server-Central" was misspelled as "Server_Central", a new location "Server_Central" will be added if the validation was turned off.
  • You may turn off validation at initial data population, or bulk data creation. This will be a saving if you let Calem add missing lookup values initially. For instance, you are upload a user with job role "Electrician", instead of manually creating "Electrician" before a user can be uploaded, you may have Calem create "Electrician" in the job role object automatically by turning off the data validation.

You can manage the data validation by adding the following line in your calem.custom.php:

//Lookup reference validation is required (true), or auto-added (false) 
Asset Item in R10
Dropdown and Lookup Customization

By accepting you will be accessing a service provided by a third-party external to