Cross-Stack RDS User Provisioning and Schema Migrations with AWS Lambda

Have you ever been in a situation where you want to provision or configure things cross-stack? Splitting these into logical stacks is always good when dealing with more complex environments. I already shared this in one of my previous blogs. But this also introduces a different problem!

Granting access to your database

When you use DynamoDB, you have to use IAM to grant access. However, other databases like MySQL also have an internal authentication method. You have a few options when you create the database in one stack and a different stack wants to access the database.

1. Use the credentials that you created at deployment time.
2. Use identity and access management (AWS IAM).
3. Create a local user per use-case.

Now, all options will work in the end, but they do have their pros and cons. I am a big fan of the least privileged principle. So, for me, using the credentials you created at deployment time is too privileged. With these credentials, you can create databases and tables. This also means that you can drop tables and delete databases. You can compare these credentials with the root credentials of a Linux system or the root account for your AWS account. The root account is there for a reason. That reason is not for day-to-day work. You should use those credentials for administrative tasks, not operational tasks.

You could use AWS IAM, and this will give us the ability to be more least privileged. This is because you need to create a user in the database. That user will receive grants on what it can do on the database and tables. You can create a read-only role if you only need to read from the database. If you also need to write, you can simply add that permission. Regardless of whether you go for option 2 or 3, you still need to provision the user and the permissions in the database.

Provisioning RDS users

After you created an RDS instance, you configured the database’s “root” credentials. The Secrets Manager should generate these credentials. Users should never have these credentials, but if they do, they are there just in case. Since we don’t want to use the root credentials, we need a user to access the database through our application. For this, we can use a provisioner lambda function. This lambda function creates the local users in the database. You can simply invoke the Lambda function as a custom resource using the same template as the RDS instance.

The Lambda function can retrieve the “root” credentials from Secrets Manager. It can use those credentials to connect to the database, create it, and load the initial schema. Next, it will create the user and the grants that the user needs. You don’t need to assign a password if you plan to use IAM. If you don’t want to use IAM, you should create the credentials in Secrets Manager and pass them dynamically into the custom resource. Afterward, your user is ready to use your application.

Provisioning RDS users cross-stack

But what if you have multiple stacks and want to use the RDS instance created in a different stack? The same principle applies. You still need to create the provisioner with the RDS instance. But instead of creating the custom resource in the template of the RDS instance, you create it in the other template. For this, you need the ARN of the Lambda function. You can use a naming schema or store the ARN in an SSM Parameter.

The advantage of this is that you can invoke the creation of database users in the template where they are used. Plus, if you make the provisioner smart enough, you can pass on the grants the user will need. This way, you encapsulate the user, the rights, and the application into the same template. This will reduce the maintenance load on your application and its infrastructure.

Schema migrations

One of the other advantages of having a provisioner is that you can also manage schema migrations. Mutating the schema can be invoked with the same ease as the user creation. This gives you the ability to perform these migrations during the CloudFormation deployments. You do need to make sure that the application is backward compatible.

Conclusion

Splitting infrastructure into multiple stacks keeps things organized, but it also introduces challenges like managing database access and schema changes. Using a Lambda function as a provisioner, you can automate user creation and schema migrations as part of your CloudFormation deployments. This approach keeps everything tightly scoped, ensures the least privilege principle, and reduces operational overhead. Whether you use IAM authentication or local users, making the provisioner smart enough to handle both ensures flexibility. With schema migrations baked into your deployments, you get a reliable and repeatable way to evolve your database without manual intervention.

Photo by Muhammed Ensar