It’s the story of couple of days back, where as usual I deployed my application on Azure App Service after testing it locally and then I was surprised when the application was not running as expected.
Since everything works locally so there were no reason to believe of any error on cloud environment too so I just do a google search and found lot of links related to this error but unfortunately none of them related to the problem I had. So with couple of my cool collogues, I started trouble shooting on this issue, and this is what all I did until I was able to drill down to the root cause.
- Tried to get the actual cause by running the dll from Azure App Service => Console option, and it tells me the error as above screenshot.
- Deployment of the application to the Azure App Service was happening through CI/CD pipeline So I thought there could be something wrong going in there hence tried the deployment through visual studio to App Service and modified the appsettings.production.json through App Service => App Service Editor option (at the time of this article this feature is on Preview) manually.
And result, it works then what the hell is going wrong with CI/CD deployment, and still no clue. - Tried the application to local IIS deployment and it worked here too. Oh my God.
- After spending hours a random thought occurred, With the working application deployed in local IIS, I replaced the appsetting.Production.json file locally by copying it from Azure App Service => App Service Editor option (to use the one deployed to azure and created by CI/CD process).
Finally here I got some relief as issue was reproducible with local env and familiar too. Hence started debugging and While debugging I found that the issue was with below code throwing exception due the keyVaultUrl value is set to Empty character.
builder.Configuration.AddAzureKeyVault(
new Uri(keyVaultUrl),
new DefaultAzureCredential(new DefaultAzureCredentialOptions
{
...
}),
new PrefixKeyVaultSecretManager(prefix));
So what was happening here is, In production, application was not using Azure Key Vault but the code was there and tried to build the configuration with AzureKeyValut and ending up throwing exception while starting the application.
CI/CD was creating the appSettings.Production.json and in this process it was replacing the keyvalut related setting to “ “ an empty character and this was the culprit.
Before I found this issue, it was looking huge but after I found it, I felt like Ahk!what the hell.
So if you ever encountered such error with production, My first recommendation would be to run the application locally with the same configuration and appsettings.json or web.config used in deployment/production and save your time. Sometime google also won’t be able to help you unless some people like me share his/her crazy experiences like this.
I too had no plan to share this :) until I got the below feedback from one of my reader for similar crazy issue related to docker networking.
Thank You All for such a beautiful feedback which turns as fuel for people like me to share more and more even it seems to be silly sometime.
Hope you enjoyed the content, follow me for more like this and please don’t forget to like/comment for it. Happy programming.