Press CTRL+K to open search

Terraform Integration Guide (Azure)

  • Published on Aug 14, 2026
  • 15 minute(s) read
Prev Next

Introduction

Lucidity manages disk lifecycles, requiring updates to the Terraform configuration to ensure seamless integration and avoid conflicts. This guide provides detailed instructions, including Azure-specific examples, to help align Terraform with Lucidity's disk management process. While the examples focus on Azure, the Terraform integration approach is adaptable and can be extended to other cloud providers, such as GCP or AWS.

We recommend involving the Lucidity support team during the terraform integration process so we can provide any tailored recommendations to ensure a smooth, streamlined workflow.

This document is mainly split into 2 sections:

  1. Onboard Lucidity for Existing MountPoints

  2. Onboard Lucidity for New MountPoints

    1. On New VMs

    2. On Existing VMs

Note: Currently, OS disks cannot be onboarded via Terraform with Lucidity. This limitation applies to both new and existing VM setups

Onboard Lucidity To Existing MountPoints

Overview

This section of the guide outlines the steps to onboard Lucidity onto already deployed VMs and disks managed by Terraform currently

Steps to Setup Lucidity on Existing MountPoints

  1. Update Terraform Code for Onboarding Lucidity
    Adjust the Terraform (TF) code to support Lucidity’s requirements.

  2. Install Lucidity Agent on the VMs
    Install the Lucidity agent directly on each VM. This can be done manually or via the Lucidity dashboard or in a scripted format based on preference.

  3. Onboard Disks to Lucidity
    Register the required disks through the Lucidity dashboard (or via APIs)

  4. Verify Functionality
    Confirm Lucidity’s functionality on the VMs, ensuring that new disks are active and original disks are no longer required

  5. Remove Original Disks from Terraform
    Once onboarded, remove references to the original disks from the Terraform configuration and remove the original disks from the infrastructure.

Terraform Code Overview

Lucidity manages two key resources in the environment, both of which need to be accounted for to avoid drift:

  • Resource Tag: ManagedByLucidity and MarkedFor

  • Disks

1. Tag Management

Lucidity applies two tags to the resources it manages: ManagedByLucidity and MarkedFor. Terraform will treat these as drift and try to remove them on every apply, so they have to be excluded.

The AzureRM provider has no provider-level tag-ignore setting. Tags must be excluded per resource, using the lifecycle block described in the next section, on every resource Lucidity tags. The provider block itself needs no changes:

Sample Configuration:

provider "azurerm" {
  features {}
}

2. Lifecycle Management

Add the ignore_changes attribute to the lifecycle block in the VM resource. This instructs Terraform to disregard any changes to tags that are being managed by Lucidity. Add this block to the disk resource or the VM block where the disk is defined.

Sample Lifecycle Block:

lifecycle {
  ignore_changes = [
    tags["ManagedByLucidity"],
    tags["MarkedFor"]
  ]
}

Note: Adjustments to this code block may vary based on how the Terraform code is structured

3. Removing Original Disks

After onboarding with Lucidity, retain the original disks temporarily as a backup. Eventually, remove them from the Terraform code directly to prevent drifts in infrastructure configuration. Consider the following when updating the code:

  • How disks and VMs are defined (together or separately managed)

  • Usage of modules and variable files

  • Any hardcoded disk references in the Terraform code

Code walkthrough

To provide practical insight into the required Terraform code changes for Lucidity onboarding, here are a few common scenarios:

Scenario 1: Data Disks Defined With the VM

This scenario covers VMs whose data disks are declared alongside the VM itself. Which variant applies depends on which resource your Terraform uses. If your VM resource is azurerm_virtual_machine and contains storage_data_disk blocks, follow 1a. If your VM resource is azurerm_linux_virtual_machine or azurerm_windows_virtual_machine, follow 1b - those resources do not support inline data disks, so your disks are already separate resources.

1a. Legacy resource: azurerm_virtual_machine

Note: azurerm_virtual_machine is deprecated in favour of azurerm_linux_virtual_machine and azurerm_windows_virtual_machine. It is documented here because existing infrastructure still uses it. New Terraform should follow 1b.

Before Lucidity Integration

Initially, the Azure VM is configured with Managed Disks, with the basic setup ensuring disks are attached directly to the VM and tagged accordingly. The following Terraform configuration demonstrates this setup:

resource "azurerm_virtual_machine" "example" {
  name                  = "MyExampleVM"
  location              = "East US"
  resource_group_name   = azurerm_resource_group.example.name
  vm_size               = "Standard_DS1_v2"
  network_interface_ids = [
    azurerm_network_interface.example.id,
  ]
  storage_os_disk {
    name              = "myExampleOsDisk"
    create_option     = "FromImage"
    caching           = "ReadWrite"
    managed_disk_type = "Premium_LRS"
  }
  storage_data_disk {
    name          = "example-datadisk1"
    create_option = "Empty"
    disk_size_gb  = 8
    lun           = 0
    caching       = "None"
  }
  storage_data_disk {
    name          = "example-datadisk2"
    create_option = "Empty"
    disk_size_gb  = 16
    lun           = 1
    caching       = "None"
  }
  tags = {
    Environment = "Production"
  }
}

In this configuration, any change to the VM or its disks would be directly managed by Terraform, including updates to tags and volume settings.

After Lucidity Integration

After integrating Lucidity, below modifications need to be made to the Terraform configuration to delegate management of specific attributes to Lucidity, such as tags and lifecycle changes for the managed disks. This ensures that Lucidity's automated processes can manage these aspects without Terraform attempting to revert them. Here is how the configuration after the changes:

resource "azurerm_virtual_machine" "example" {
  name                  = "MyExampleVM"
  location              = "East US"
  resource_group_name   = azurerm_resource_group.example.name
  vm_size               = "Standard_DS1_v2"
  network_interface_ids = [
    azurerm_network_interface.example.id,
  ]
  storage_os_disk {
    name              = "myExampleOsDisk"
    create_option     = "FromImage"
    caching           = "ReadWrite"
    managed_disk_type = "Premium_LRS"
  }
  tags = {
    Environment = "Production"
  }
  lifecycle {
    ignore_changes = [
      storage_data_disk, // Lucidity manages disk lifecycle
      tags["MarkedFor"],
      tags["ManagedByLucidity"] // Lucidity Tags
    ]
  }
}

Key changes

Note, the key change here is the Lifecycle management change using ignore_changes attribute which tells Terraform to ignore any changes to the tags managed by Lucidity.

Note: All disks on the VM will get ignored by TF as per the example in the code. If all disks are not onboarded, this needs to be noted.

1b. Current resources: azurerm_linux_virtual_machine

On the current VM resources, data disks are always separate azurerm_managed_disk resources joined by azurerm_virtual_machine_data_disk_attachment. The same applies to azurerm_windows_virtual_machine.

Before Lucidity Integration

resource "azurerm_linux_virtual_machine" "example" {
  name                  = "MyExampleVM"
  location              = "East US"
  resource_group_name   = azurerm_resource_group.example.name
  size                  = "Standard_DS1_v2"
  admin_username        = "adminuser"
  network_interface_ids = [azurerm_network_interface.example.id]
  os_disk {
    caching              = "ReadWrite"
    storage_account_type = "Premium_LRS"
  }
  tags = {
    Environment = "Production"
  }
}

resource "azurerm_managed_disk" "datadisk1" {
  name                 = "example-datadisk1"
  location             = "East US"
  resource_group_name  = azurerm_resource_group.example.name
  storage_account_type = "Premium_LRS"
  create_option        = "Empty"
  disk_size_gb         = 8
}

resource "azurerm_virtual_machine_data_disk_attachment" "datadisk1" {
  managed_disk_id    = azurerm_managed_disk.datadisk1.id
  virtual_machine_id = azurerm_linux_virtual_machine.example.id
  lun                = 0
  caching            = "None"
}

After Lucidity Integration

The onboarded disk and its attachment are removed from the configuration, and the VM gets a lifecycle block for the Lucidity tags. 

resource "azurerm_linux_virtual_machine" "example" {
  name                  = "MyExampleVM"
  location              = "East US"
  resource_group_name   = azurerm_resource_group.example.name
  size                  = "Standard_DS1_v2"
  admin_username        = "adminuser"
  network_interface_ids = [azurerm_network_interface.example.id]
  os_disk {
    caching              = "ReadWrite"
    storage_account_type = "Premium_LRS"
  }
  tags = {
    Environment = "Production"
  }
  lifecycle {
    ignore_changes = [
      tags["ManagedByLucidity"],
      tags["MarkedFor"]
    ]
  }
}

// azurerm_managed_disk.datadisk1 and its attachment are removed -
// that disk is now managed by Lucidity. Remove them from state too:
//   terraform state rm azurerm_managed_disk.datadisk1
//   terraform state rm azurerm_virtual_machine_data_disk_attachment.datadisk1

Scenario 2: Externally Managed Disks

This scenario details configuring managed disks that are not directly attached during VM creation. Instead, they are managed separately, allowing flexibility for detaching or reattaching disks without affecting the VM lifecycle. After Lucidity integration, additional configuration is required to facilitate Lucidity's disk management.

Before Lucidity Integration

Initially, managed disks are defined as separate resources and attached to VMs through attachment specifications in Terraform. Here's how you might define this in your Terraform script before integrating with Lucidity:

resource "azurerm_managed_disk" "example_disk_0" {
  name                 = "exampleDisk0"
  location             = "East US"
  resource_group_name  = azurerm_resource_group.example.name
  storage_account_type = "Standard_LRS"
  create_option        = "Empty"
  disk_size_gb         = 50
}

resource "azurerm_virtual_machine_data_disk_attachment" "example_attach_0" {
  managed_disk_id    = azurerm_managed_disk.example_disk_0.id
  virtual_machine_id = azurerm_linux_virtual_machine.example.id
  lun                = 0
  caching            = "ReadWrite"
}

resource "azurerm_managed_disk" "example_disk_1" {
  name                 = "exampleDisk1"
  location             = "East US"
  resource_group_name  = azurerm_resource_group.example.name
  storage_account_type = "Standard_LRS"
  create_option        = "Empty"
  disk_size_gb         = 100
}

resource "azurerm_virtual_machine_data_disk_attachment" "example_attach_1" {
  managed_disk_id    = azurerm_managed_disk.example_disk_1.id
  virtual_machine_id = azurerm_linux_virtual_machine.example.id
  lun                = 1
  caching            = "ReadWrite"
}

After Lucidity Integration

After Lucidity is integrated, the below modifications need to be made to ensure that changes to tags and volume_tags are ignored by Terraform. This prevents Terraform from attempting to manage or revert these properties, allowing Lucidity to handle them:

resource "azurerm_managed_disk" "example_disk_1" {
  name                 = "exampleDisk1"
  location             = "East US"
  resource_group_name  = azurerm_resource_group.example.name
  storage_account_type = "Standard_LRS"
  create_option        = "Empty"
  disk_size_gb         = 100

  lifecycle {
    ignore_changes = [
      tags["MarkedFor"],
      tags["ManagedByLucidity"] // Lucidity Tags 
    ]
  }
}

resource "azurerm_virtual_machine_data_disk_attachment" "example_attach_1" {
  managed_disk_id    = azurerm_managed_disk.example_disk_1.id
  virtual_machine_id = azurerm_linux_virtual_machine.example.id
  lun                = 1
  caching            = "ReadWrite"
}

State Management

Here note, users need to manually remove the managed disk and disk attachment blocks from the Terraform configuration files when these resources are no longer managed by Terraform

Users must also remove the corresponding state file data using Terraform commands:

terraform state rm azurerm_managed_disk.example_disk_0
terraform state rm azurerm_virtual_machine_data_disk_attachment.example_attach_0

  • As an alternative to manually editing the state file and Terraform configuration, users can use the following command to refresh the state based on the actual infrastructure, effectively accepting any changes made outside of Terraform after disks are removed:

terraform apply -refresh-only -auto-approve

Scenario 3: Modules and Variable Files for Disk Management

In this scenario, Terraform modules and variable files are used to create and manage disks independently of the VMs. When Lucidity takes over certain aspects of disk management, adjustments are needed within these modules and variables to ensure compatibility.

Before Lucidity Integration

Initially, development teams use the modules by setting the required variables in terraform.tfvars files. An example of this setup might look like:

# In terraform.tfvars file

disk_details = {
  "disk0" = {
    size = 8
    type = "Standard_LRS"
  },
  "disk1" = {
    size = 16
    type = "Premium_LRS"
  }
}

# In main.tf file

module "managed_disks" {
  source        = "./modules/managed_disks"
  disks = var.disk_details
}

After Lucidity Integration

After integrating with Lucidity, it's important to ensure the module is adjusted and variable files are updated to reflect the changes. Assuming we are onboarding disk1 the code would look like:

# In terraform.tfvars file

disk_details = {
  "disk0" = {
    size = 8
    type = "Standard_LRS"
  }
}

// disk1 removed

# In main.tf file

module "managed_disks" {
  source = "./modules/managed_disks"
  disks  = var.disk_details
}

# Assuming the managed_disks module defines resources like this:
resource "azurerm_managed_disk" "example" {
  for_each             = var.disks
  name                 = "ManagedDisk-${each.key}"
  location             = "East US"
  resource_group_name  = azurerm_resource_group.example.name
  storage_account_type = each.value.type
  create_option        = "Empty"
  disk_size_gb         = each.value.size

  lifecycle {
    ignore_changes = [
      tags["MarkedFor"],
      tags["ManagedByLucidity"] //Tags are managed outside of Terraform
    ]
  }
}

Key Operations

  1. Modify the Variable File:

  • Update terraform.tfvars to match the resources and parameters managed by Terraform.

  1. Update the Module Configuration:

  • If needed, add or modify lifecycle blocks within your modules to prevent Terraform from attempting to manage aspects now handled by Lucidity.

  1. State Reconciliation:

  • Use terraform apply -refresh-only -auto-approve to update the Terraform state if Lucidity has made changes to the resources.

These 3 scenarios illustrate how Terraform configurations adapt to Lucidity integration across various setups. Each scenario is aimed at maintaining seamless infrastructure management while leveraging Lucidity’s disk management capabilities.

Onboard Lucidity On New MountPoints

Overview

This section of the guide outlines the steps to onboard Lucidity onto new VMs being deployed to the infrastructure using TerraForm or new partitions being added to existing VMs using TerraForm. Lucidity handles the disk management on its end and hence, requires updates to the Terraform configuration to prevent conflicts and manage disk lifecycles.

The process involves configuring the Terraform script to include necessary parameters and making an API call to Lucidity’s dashboard backend. This results in the creation of a new partition from an existing disk pool, which Lucidity then manages

Steps to Setup Lucidity For a New MountPoint

New VM:

  1. Create a new VM using Terraform
    Create a new VM using terraform however do NOT add the disks you want to onboard to Lucidity yet.

  2. Install Lucidity Agent
    Install the Lucidity agent on the VM. This can be done manually or via the Lucidity dashboard or in a scripted format based on preference.

  3. Modify Terraform Code to support Lucidity
    Adjust the Terraform code to support Lucidity

  4. Onboard Disks via Terraform
    Add the new partition using Terraform as described by the next Code walkthrough section.

  5. Verify Functionality
    As a check, verify disk has been onboarded via the dashboard and by logging into the VM.

Existing VM:

If the VM already has disks onboarded to Lucidity and you are looking to add more disks to the same VM:

  1. Steps 1 - 3
    Steps 1 to step 3 (from the previous section) should already be completed since VM has already been onboarded however we recommend verifying the same.

  2. Onboard Disks via Terraform
    Adjust the Terraform code to add the new partition as described by the next section.

  3. Ensure disk configurations matches existing Lucidity disks
    When defining the disk, the new disk host cache setting, disk type and tags have to be the same as existing Lucidity disks.

  4. Verify Functionality
    As a check, verify disk has been onboarded via the dashboard and by logging into the VM.

Terraform Code Walkthrough

Ensure that the Provider Block has been updated to support onboarding Lucidity (as defined in previous sections).

Ensure that the Lifecycle Block has been updated to support onboarding Lucidity (as defined in previous sections).

Since no original disks need to be removed, no changes needed to the Terraform code for existing disks.

Users only need to modify their existing Terraform scripts to automate the partition creation and onboarding process for the new disk that is to be added. The script will include details like the partition name, instance ID, disk type, and Azure-specific settings such as disk cache options.

Here is a sample Terraform script tailored for Azure (Windows):

resource "null_resource" "create_new_mount_instance_windows" {
  provisioner "local-exec" {
    on_failure  = fail
    interpreter = ["PowerShell", "-Command"]
    command     = <<-EOT
    $uri = "http://<dashboardurl>/api/v1/partition/create"
    $headers = @{
        "Authorization" = "secretkey"
        "X-Authtype" = "auth_key"
        "X-Tenant" = "<tenantId>"

        "X-Account" = "<accountid>"
        "accept" = "*/*"
        "Content-Type" = "application/json"
        "access-id" = "accesskey"
    }
    $body = @{
        "diskType" = "Standard_LRS"  # Azure specific disk type
        "instance" = "<instanceid>"
        "partition" = "E"  # New partition to be created
        "tenant" = "<tenantId>"
        "diskCache" = "ReadWrite"  # Azure specific cache setting

        "azureCmkDiskEncryptionSetId" = "/subscriptions/<sub>/resourceGroups/<rgrp>/providers/Microsoft.Compute/diskEncryptionSets/<cmkId>" 

 "azureCmkPmkDiskEncryptionSetId" = "/subscriptions/<sub>/resourceGroups/<rgrp>/providers/Microsoft.Compute/diskEncryptionSets/<pmkId>"
    } | ConvertTo-Json

    Invoke-RestMethod -Uri $uri -Method Post -Headers $headers -Body $body
    EOT
  }
  triggers = {
    always_run = timestamp()
  }
}

Here is a sample Terraform script tailored for Azure (Linux):

resource "null_resource" "create_new_mount_instance_azure" {

  provisioner "local-exec" {
    command = <<-EOT
    #!/bin/bash
    uri="http://<dashboardurl>/api/v1/partition/create"
    headers=(
        -H "Authorization: secretkey"
        -H "X-Authtype: auth_key"
        -H "X-Tenants: <tenantId>"
        -H "X-Tenant: <tenantId>"

        -H "X-Account: <accountid>"
        -H "accept: */*"
        -H "Content-Type: application/json"
        -H "access-id: accesskey"
    )
    body=$(jq -n \
        --arg diskType "<diskType>" \
        --arg instance "<instanceid>" \
        --arg partition "J" \
        --arg tenant "<tenantId>" \
        --arg azureCmkDiskEncryptionSetId "/subscriptions/<sub>/resourceGroups/<rgrp>/providers/Microsoft.Compute/diskEncryptionSets/<cmkId>" \
        --arg azureCmkPmkDiskEncryptionSetId "/subscriptions/<sub>/resourceGroups/<rgrp>/providers/Microsoft.Compute/diskEncryptionSets/<pmkId>" \
        '{
            diskType: $diskType,
            instance: $instance,
            partition: $partition,
            tenant: $tenant,
            azureCmkDiskEncryptionSetId: $azureCmkDiskEncryptionSetId,
            azureCmkPmkDiskEncryptionSetId: $azureCmkPmkDiskEncryptionSetId
        }')

    curl -X POST "${headers[@]}" -d "$body" "$uri"
    EOT
  }

  triggers = {
    always_run = timestamp()
  }
}

Parameters and Options:

  • diskType: Users should provide the disk type appropriate for their use case. Azure options include Premium_LRS, Standard_LRS, etc.

  • diskCache: For Azure, options like "None", "ReadOnly", and "ReadWrite" allow customization of how the disk interacts with cached data.

  • instance, tenant, and partition: Mandatory fields to specify the exact resource being managed.

  • azureCmkDiskEncryptionSetId and azureCmkPmkDiskEncryptionSetId: These fields refer to the customer-managed keys and platform-managed keys. These IDs link to the Azure disk encryption sets that encrypt the disk partitions. Users need to provide these if they have specific encryption requirements or default settings will be applied if left empty.

Integration Process:

Upon executing the updated Terraform script, an API call is made to Lucidity’s backend, which handles the creation and immediate integration of the new partition. This ensures that the new partition is set up for automated management actions, such as capacity adjustments based on usage.

Related articles