# File Job ### File Job A **File Job** is an automated task that manages files **locally on your Edge device**, allowing you to copy, move, or extract files between storage containers or folders. File Jobs are ideal for organizing, processing, or preparing files before they are ingested, transferred, or archived - completely independent of external servers or network shares. File Jobs operate on the built-in Edge storage of your device, which is powered by a **Blob Storage Module** (offering functionality similar to [Azure Storage Accounts](https://learn.microsoft.com/en-us/azure/storage/common/storage-account-overview)). This enables robust and flexible file handling, even during network outages. *** ### Key Features * **Edge-Local Operation:** Runs entirely on the Edge device, ensuring fast, reliable file movement with no dependency on external connectivity. * **Flexible Scheduling:** Define how often the job runs using cron-style expressions or simple intervals. * **Container/Folders:** Move, copy, or extract files between any storage containers or folders configured on the Edge device. * **Pattern Matching:** Select files to handle based on patterns (e.g., all files in any subfolder use `**/*`, or target files of a specific extension in the import folder and any sub-folder where file extension is .csv use `import/**/*.csv`). * **Zip Extraction:** Automatically extract files from zip archives as part of your workflow. * **File Operations:** Choose to copy or move files, preserving or removing the originals as needed. #### Typical Use Cases * **Staging Files:** Prepare and organize files collected from lab instruments or production systems before they are transferred to the cloud or another destination. * **Automated Extraction:** Unzip archived measurement files for further processing or ingestion. * **Data Preparation:** Filter and sort files into different containers or folders for downstream applications or analytics. * **Edge Processing:** Enable local processing of files for performance, compliance, or offline operations. *** ### Configuring a File Job 1. **Add a New Job** * Select the Area node where you want to add the job. * Go to the **Jobs** tab. * Click **+ New job** and select **File job** and provide a **name** for the job.

Create a File Job

2. **Configure Job Settings**

Configuration of File Job

* **Job Name:** Provide a descriptive name (e.g., EdgeFileJob). * **Enabled:** Toggle to activate or deactivate the job. * **Schedule (CRON):** Define how frequently the job should run (e.g., every 10 minutes: `0/10 * * * *`). > The CRON string consists of 5 characters separated by spaces, describing how often the schedule should be triggered on the Edge device. > > **Examples:** > > * `* * * * *` Run every minute > * `*/30 * * * *` Run every 30 minutes > * `0 * * * *` Run every hour > * `0 0 * * *` Run once a day at midnight UTC > * `0 0 1 * *` Run on the 1st of each month at midnight UTC > * `0 0 * * 1` Run at midnight every Monday UTC > > [See cron.help for more details.](https://cron.help/) 3. **Source** * **Source Container Name:** The name of the storage container or folder where the files are located (e.g., `tmplabfiles`). * **Source Folder:** (Optional) Specify a subfolder if needed. Eg. `folder1` or `folder1/folder2` to create a deeper directory structure. * **File Patterns:** Define patterns to target files to handle using file *system globbing patterns*, you can define rules for files that should be targeted for the operation. (e.g., `**/*` for all files in any subdirectory). > You can use multiple rules by seperating them with **`;;`**\ > \ > Eg. `(filepattern1)`;;`(filepattern2)`;;`(filepattern3)`\ > \ > Here are some filepattern examples:
> > * `*.txt` Matches all files with the file extension .txt in the folder. > * `*.*` Matches all files regardless of file extension in the current folder. > * `*word*` Matches all files that has the name 'word' in the filename. > * `styles/*.css` Matches all files with extension '.css' in the directory 'styles/'. > * `scripts/*/*` Matches all files in 'scripts/' or one level of subdirectory under 'scripts/' > * `images*/*` Matches all files in a folder with name that is or begins with 'images'. > * `**/*` Matches all files in any subdirectory. > * `dir/**/*` Matches all files in any subdirectory under 'dir/'. > * `GTRS*/**/*` Matches all files, that has a starting directory that starts with 'GTRS' and include all files from any subdirectory. > > For more information see [Microsoft documentation](https://docs.microsoft.com/en-us/dotnet/api/microsoft.extensions.filesystemglobbing.matcher?view=dotnet-plat-ext-5.0) which include more examples. * **Extract Zip Files:** Enable to automatically extract files with a `.zip` extension as part of the job. * **File Operation:** Choose between `Copy` (preserve originals) or `Move` (remove from source after processing). 4. **Destination** * **Destination Container Name:** Target storage container for the moved/copied files (e.g., `unzipped`). * **Destination Folder:** (Optional) Subfolder for organization. Eg. `folder1` or `folder1/folder2` to create a deeper directory structure.\ You can also use tokens to build up the path using the tokens below:
| $(deviceid) | Replace a part of a filepath with the Device Id of the Edge device using the token $(deviceid). |

rootfolder/$(deviceid)/otherfolder

rootfolder/EdgeDevice-1/otherfolder

| | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------ | | $(yyyy) | Replace a part of a filepath with the current Year UTC (Eg. 2021) using the token $(yyyy). |

rootfolder/$(yyyy)/otherfolder

rootfolder/2021/otherfolder

| | $(MM) | Replace a part of a filepath with the current Month UTC (Eg. 01-12) using the token $(MM). |

rootfolder/$(MM)/otherfolder

rootfolder/07/otherfolder

| | $(dd) | Replace a part of a filepath with the current Day of month UTC (Eg. 01-31) using the token $(dd). |

rootfolder/$(dd)/otherfolder

rootfolder/07/otherfolder

| | $(HH) | Replace a part of a filepath with the current UTC hour (Eg. 00-24) using the token $(HH). |

rootfolder/$(HH)/otherfolder

rootfolder/13/otherfolder

| | $(mm) | Replace a part of a filepath with the current UTC minute (Eg. 00-59) using the token $(mm). |

rootfolder/$(mm)/otherfolder

rootfolder/57/otherfolder

| | $(ss) | Replace a part of a filepath with the current UTC Second (Eg. 00-59) using the token $(ss). |

rootfolder/$(ss)/otherfolder

rootfolder/35/otherfolder

| | $(FFF) | Replace a part of a filepath with the current UTC MilliSecond (Eg. 000-999) using the token $(FFF). |

rootfolder/$(FFF)/otherfolder

rootfolder/358/otherfolder

| | $(unixts) | Replace a part of a filepath with the current Unix Timestamp UTC (Number of milliseconds that has elapsed since 1970-01-01) using the token $(unixts). |

rootfolder/$(unixts)/otherfolder

rootfolder/1629098246320/otherfolder

| *** ### Example File Job Configuration | Setting | Value | Example | | -------------------------- | ------------------- | ------------- | | Job Name | EdgeFileJob | EdgeFileJob | | Enabled | Yes | | | CRON Schedule | Every hour | 0 \* \* \* \* | | Source Container Name | tmplabfiles | tmplabfiles | | Source Folder | (empty) | | | File Patterns | All files | \*\**/\** | | Extract Zip Files | Yes/No | Yes | | File Operation | Move | Move | | Destination Container Name | unzipped | unzipped | | Destination Folder | Name of Edge device | $(deviceid) | *** ### Best Practices * **Use clear names** for jobs, containers, and folders to simplify troubleshooting and management. * **Schedule jobs appropriately** to match your workflow and avoid unnecessary load on the Edge device. * **Test with sample files** to verify pattern matching and operations before deploying in production. * **Remember:** File Jobs cannot interact with external systems- use FTP Jobs or File Share Jobs for networked file transfers. * **Monitor File Jobs** Each time the File Job triggers, it will create a log entry in the File Transfer Module. You can monitor all Job Logs in the [FileTransferModule Jobs Tab](https://docs.tricloudnexus.io/management-portal/operations/monitoring/module-details/jobs-history) (the Asset Hierarchy containing the Job must be deployed first).