diff --git a/docs/features/configuration.rst b/docs/features/configuration.rst index 4e11e7ed..220ae2bc 100644 --- a/docs/features/configuration.rst +++ b/docs/features/configuration.rst @@ -1,174 +1,193 @@ -Configuration -============ - -An example configuration can be found `here `_. -There are two sections to the configuration. An array of ReRoutes and a GlobalConfiguration. -The ReRoutes are the objects that tell Ocelot how to treat an upstream request. The Global -configuration is a bit hacky and allows overrides of ReRoute specific settings. It's useful -if you don't want to manage lots of ReRoute specific settings. - -.. code-block:: json - - { - "ReRoutes": [], - "GlobalConfiguration": {} - } - -Here is an example ReRoute configuration, You don't need to set all of these things but this is everything that is available at the moment: - -.. code-block:: json - - { - "DownstreamPathTemplate": "/", - "UpstreamPathTemplate": "/", - "UpstreamHttpMethod": [ - "Get" - ], - "AddHeadersToRequest": {}, - "AddClaimsToRequest": {}, - "RouteClaimsRequirement": {}, - "AddQueriesToRequest": {}, - "RequestIdKey": "", - "FileCacheOptions": { - "TtlSeconds": 0, - "Region": "" - }, - "ReRouteIsCaseSensitive": false, - "ServiceName": "", - "DownstreamScheme": "http", - "DownstreamHostAndPorts": [ - { - "Host": "localhost", - "Port": 51876, - } - ], - "QoSOptions": { - "ExceptionsAllowedBeforeBreaking": 0, - "DurationOfBreak": 0, - "TimeoutValue": 0 - }, - "LoadBalancer": "", - "RateLimitOptions": { - "ClientWhitelist": [], - "EnableRateLimiting": false, - "Period": "", - "PeriodTimespan": 0, - "Limit": 0 - }, - "AuthenticationOptions": { - "AuthenticationProviderKey": "", - "AllowedScopes": [] - }, - "HttpHandlerOptions": { - "AllowAutoRedirect": true, - "UseCookieContainer": true, - "UseTracing": true - }, - "UseServiceDiscovery": false, - "DangerousAcceptAnyServerCertificateValidator": false - } - -More information on how to use these options is below.. - -Multiple environments -^^^^^^^^^^^^^^^^^^^^^ - -Like any other asp.net core project Ocelot supports configuration file names such as configuration.dev.json, configuration.test.json etc. In order to implement this add the following -to you - -.. code-block:: csharp - - .ConfigureAppConfiguration((hostingContext, config) => - { - config - .SetBasePath(hostingContext.HostingEnvironment.ContentRootPath) - .AddJsonFile("appsettings.json", true, true) - .AddJsonFile($"appsettings.{hostingContext.HostingEnvironment.EnvironmentName}.json", true, true) - .AddJsonFile("ocelot.json") - .AddJsonFile($"configuration.{hostingContext.HostingEnvironment.EnvironmentName}.json") - .AddEnvironmentVariables(); - }) - -Ocelot will now use the environment specific configuration and fall back to ocelot.json if there isnt one. - -You also need to set the corresponding environment variable which is ASPNETCORE_ENVIRONMENT. More info on this can be found in the `asp.net core docs `_. - -Merging configuration files -^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -This feature was requested in `Issue 296 `_ and allows users to have multiple configuration files to make managing large configurations easier. - -Instead of adding the configuration directly e.g. AddJsonFile("ocelot.json") you can call AddOcelot() like below. - -.. code-block:: csharp - - .ConfigureAppConfiguration((hostingContext, config) => - { - config - .SetBasePath(hostingContext.HostingEnvironment.ContentRootPath) - .AddJsonFile("appsettings.json", true, true) - .AddJsonFile($"appsettings.{hostingContext.HostingEnvironment.EnvironmentName}.json", true, true) - .AddOcelot() - .AddEnvironmentVariables(); - }) - -In this scenario Ocelot will look for any files that match the pattern (?i)ocelot.([a-zA-Z0-9]*).json and then merge these together. If you want to set the GlobalConfiguration property you must have a file called ocelot.global.json. - -The way Ocelot merges the files is basically load them, loop over them, add any ReRoutes, add any AggregateReRoutes and if the file is called ocelot.global.json add the GlobalConfiguration aswell as any ReRoutes or AggregateReRoutes. Ocelot will then save the merged configuration to a file called ocelot.json and this will be used as the source of truth while ocelot is running. - -At the moment there is no validation at this stage it only happens when Ocelot validates the final merged configuration. This is something to be aware of when you are investigating problems. I would advise always checking what is in ocelot.json if you have any problems. - -Store configuration in consul -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -If you add the following when you register your services Ocelot will attempt to store and retrieve its configuration in consul KV store. - -.. code-block:: csharp - - services - .AddOcelot() - .AddStoreOcelotConfigurationInConsul(); - -You also need to add the following to your ocelot.json. This is how Ocelot -finds your Consul agent and interacts to load and store the configuration from Consul. - -.. code-block:: json - - "GlobalConfiguration": { - "ServiceDiscoveryProvider": { - "Host": "localhost", - "Port": 9500 - } - } - -I decided to create this feature after working on the raft consensus algorithm and finding out its super hard. Why not take advantage of the fact Consul already gives you this! -I guess it means if you want to use Ocelot to its fullest you take on Consul as a dependency for now. - -This feature has a 3 second ttl cache before making a new request to your local consul agent. - -Follow Redirects / Use CookieContainer -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Use HttpHandlerOptions in ReRoute configuration to set up HttpHandler behavior: - -1. AllowAutoRedirect is a value that indicates whether the request should follow redirection responses. Set it true if the request should automatically -follow redirection responses from the Downstream resource; otherwise false. The default value is false. -2. UseCookieContainer is a value that indicates whether the handler uses the CookieContainer -property to store server cookies and uses these cookies when sending requests. The default value is false. Please note -that if you are using the CookieContainer Ocelot caches the HttpClient for each downstream service. This means that all requests -to that DownstreamService will share the same cookies. `Issue 274 `_ was created because a user -noticed that the cookies were being shared. I tried to think of a nice way to handle this but I think it is impossible. If you don't cache the clients -that means each request gets a new client and therefore a new cookie container. If you clear the cookies from the cached client container you get race conditions due to inflight -requests. This would also mean that subsequent requests dont use the cookies from the previous response! All in all not a great situation. I would avoid setting -UseCookieContainer to true unless you have a really really good reason. Just look at your response headers and forward the cookies back with your next request! - -SSL Errors -^^^^^^^^^^ - -Id you want to ignore SSL warnings / errors set the following in your ReRoute config. - -.. code-block:: json - - "DangerousAcceptAnyServerCertificateValidator": false - -I don't reccomend doing this, I suggest creating your own certificate and then getting it trusted by your local / remote machine if you can. \ No newline at end of file +Configuration +============ + +An example configuration can be found `here `_. +There are two sections to the configuration. An array of ReRoutes and a GlobalConfiguration. +The ReRoutes are the objects that tell Ocelot how to treat an upstream request. The Global +configuration is a bit hacky and allows overrides of ReRoute specific settings. It's useful +if you don't want to manage lots of ReRoute specific settings. + +.. code-block:: json + + { + "ReRoutes": [], + "GlobalConfiguration": {} + } + +Here is an example ReRoute configuration, You don't need to set all of these things but this is everything that is available at the moment: + +.. code-block:: json + + { + "DownstreamPathTemplate": "/", + "UpstreamPathTemplate": "/", + "UpstreamHttpMethod": [ + "Get" + ], + "AddHeadersToRequest": {}, + "AddClaimsToRequest": {}, + "RouteClaimsRequirement": {}, + "AddQueriesToRequest": {}, + "RequestIdKey": "", + "FileCacheOptions": { + "TtlSeconds": 0, + "Region": "" + }, + "ReRouteIsCaseSensitive": false, + "ServiceName": "", + "DownstreamScheme": "http", + "DownstreamHostAndPorts": [ + { + "Host": "localhost", + "Port": 51876, + } + ], + "QoSOptions": { + "ExceptionsAllowedBeforeBreaking": 0, + "DurationOfBreak": 0, + "TimeoutValue": 0 + }, + "LoadBalancer": "", + "RateLimitOptions": { + "ClientWhitelist": [], + "EnableRateLimiting": false, + "Period": "", + "PeriodTimespan": 0, + "Limit": 0 + }, + "AuthenticationOptions": { + "AuthenticationProviderKey": "", + "AllowedScopes": [] + }, + "HttpHandlerOptions": { + "AllowAutoRedirect": true, + "UseCookieContainer": true, + "UseTracing": true + }, + "UseServiceDiscovery": false, + "DangerousAcceptAnyServerCertificateValidator": false + } + +More information on how to use these options is below.. + +Multiple environments +^^^^^^^^^^^^^^^^^^^^^ + +Like any other asp.net core project Ocelot supports configuration file names such as configuration.dev.json, configuration.test.json etc. In order to implement this add the following +to you + +.. code-block:: csharp + + .ConfigureAppConfiguration((hostingContext, config) => + { + config + .SetBasePath(hostingContext.HostingEnvironment.ContentRootPath) + .AddJsonFile("appsettings.json", true, true) + .AddJsonFile($"appsettings.{hostingContext.HostingEnvironment.EnvironmentName}.json", true, true) + .AddJsonFile("ocelot.json") + .AddJsonFile($"configuration.{hostingContext.HostingEnvironment.EnvironmentName}.json") + .AddEnvironmentVariables(); + }) + +Ocelot will now use the environment specific configuration and fall back to ocelot.json if there isnt one. + +You also need to set the corresponding environment variable which is ASPNETCORE_ENVIRONMENT. More info on this can be found in the `asp.net core docs `_. + +Merging configuration files +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +This feature was requested in `Issue 296 `_ and allows users to have multiple configuration files to make managing large configurations easier. + +Instead of adding the configuration directly e.g. AddJsonFile("ocelot.json") you can call AddOcelot() like below. + +.. code-block:: csharp + + .ConfigureAppConfiguration((hostingContext, config) => + { + config + .SetBasePath(hostingContext.HostingEnvironment.ContentRootPath) + .AddJsonFile("appsettings.json", true, true) + .AddJsonFile($"appsettings.{hostingContext.HostingEnvironment.EnvironmentName}.json", true, true) + .AddOcelot() + .AddEnvironmentVariables(); + }) + +In this scenario Ocelot will look for any files that match the pattern (?i)ocelot.([a-zA-Z0-9]*).json and then merge these together. If you want to set the GlobalConfiguration property you must have a file called ocelot.global.json. + +The way Ocelot merges the files is basically load them, loop over them, add any ReRoutes, add any AggregateReRoutes and if the file is called ocelot.global.json add the GlobalConfiguration aswell as any ReRoutes or AggregateReRoutes. Ocelot will then save the merged configuration to a file called ocelot.json and this will be used as the source of truth while ocelot is running. + +At the moment there is no validation at this stage it only happens when Ocelot validates the final merged configuration. This is something to be aware of when you are investigating problems. I would advise always checking what is in ocelot.json if you have any problems. + +Store configuration in consul +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +If you add the following when you register your services Ocelot will attempt to store and retrieve its configuration in consul KV store. + +.. code-block:: csharp + + services + .AddOcelot() + .AddStoreOcelotConfigurationInConsul(); + +You also need to add the following to your ocelot.json. This is how Ocelot +finds your Consul agent and interacts to load and store the configuration from Consul. + +.. code-block:: json + + "GlobalConfiguration": { + "ServiceDiscoveryProvider": { + "Host": "localhost", + "Port": 9500 + } + } + +I decided to create this feature after working on the raft consensus algorithm and finding out its super hard. Why not take advantage of the fact Consul already gives you this! +I guess it means if you want to use Ocelot to its fullest you take on Consul as a dependency for now. + +This feature has a 3 second ttl cache before making a new request to your local consul agent. + +Configuration Key +----------------- + +If you are using Consul for configuration (or other providers in the future) you might want to key your configurations so you can have multiple configurations :) This feature was requested in `issue 346 `_! In order to specify the key you need to set the ConfigurationKey property in the ServiceDiscoveryProvider section of the configuration json file e.g. + +.. code-block:: json + + "GlobalConfiguration": { + "ServiceDiscoveryProvider": { + "Host": "localhost", + "Port": 9500, + "ConfigurationKey": "Oceolot_A" + } + } + +In this example Ocelot will use Oceolot_A as the key for your configuration when looking it up in Consul. + +If you do not set the ConfigurationKey Ocelot will use the string InternalConfiguration as the key. + +Follow Redirects / Use CookieContainer +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Use HttpHandlerOptions in ReRoute configuration to set up HttpHandler behavior: + +1. AllowAutoRedirect is a value that indicates whether the request should follow redirection responses. Set it true if the request should automatically +follow redirection responses from the Downstream resource; otherwise false. The default value is false. +2. UseCookieContainer is a value that indicates whether the handler uses the CookieContainer +property to store server cookies and uses these cookies when sending requests. The default value is false. Please note +that if you are using the CookieContainer Ocelot caches the HttpClient for each downstream service. This means that all requests +to that DownstreamService will share the same cookies. `Issue 274 `_ was created because a user +noticed that the cookies were being shared. I tried to think of a nice way to handle this but I think it is impossible. If you don't cache the clients +that means each request gets a new client and therefore a new cookie container. If you clear the cookies from the cached client container you get race conditions due to inflight +requests. This would also mean that subsequent requests dont use the cookies from the previous response! All in all not a great situation. I would avoid setting +UseCookieContainer to true unless you have a really really good reason. Just look at your response headers and forward the cookies back with your next request! + +SSL Errors +^^^^^^^^^^ + +Id you want to ignore SSL warnings / errors set the following in your ReRoute config. + +.. code-block:: json + + "DangerousAcceptAnyServerCertificateValidator": false + +I don't reccomend doing this, I suggest creating your own certificate and then getting it trusted by your local / remote machine if you can. diff --git a/src/Ocelot/Configuration/Builder/ServiceProviderConfigurationBuilder.cs b/src/Ocelot/Configuration/Builder/ServiceProviderConfigurationBuilder.cs index cd8446a5..25df651c 100644 --- a/src/Ocelot/Configuration/Builder/ServiceProviderConfigurationBuilder.cs +++ b/src/Ocelot/Configuration/Builder/ServiceProviderConfigurationBuilder.cs @@ -1,39 +1,46 @@ -namespace Ocelot.Configuration.Builder -{ - public class ServiceProviderConfigurationBuilder - { - private string _serviceDiscoveryProviderHost; - private int _serviceDiscoveryProviderPort; - private string _type; - private string _token; - - public ServiceProviderConfigurationBuilder WithHost(string serviceDiscoveryProviderHost) - { - _serviceDiscoveryProviderHost = serviceDiscoveryProviderHost; - return this; - } - - public ServiceProviderConfigurationBuilder WithPort(int serviceDiscoveryProviderPort) - { - _serviceDiscoveryProviderPort = serviceDiscoveryProviderPort; - return this; - } - - public ServiceProviderConfigurationBuilder WithType(string type) - { - _type = type; - return this; - } - - public ServiceProviderConfigurationBuilder WithToken(string token) - { - _token = token; - return this; - } - - public ServiceProviderConfiguration Build() - { - return new ServiceProviderConfiguration(_type, _serviceDiscoveryProviderHost, _serviceDiscoveryProviderPort, _token); - } - } -} +namespace Ocelot.Configuration.Builder +{ + public class ServiceProviderConfigurationBuilder + { + private string _serviceDiscoveryProviderHost; + private int _serviceDiscoveryProviderPort; + private string _type; + private string _token; + private string _configurationKey; + + public ServiceProviderConfigurationBuilder WithHost(string serviceDiscoveryProviderHost) + { + _serviceDiscoveryProviderHost = serviceDiscoveryProviderHost; + return this; + } + + public ServiceProviderConfigurationBuilder WithPort(int serviceDiscoveryProviderPort) + { + _serviceDiscoveryProviderPort = serviceDiscoveryProviderPort; + return this; + } + + public ServiceProviderConfigurationBuilder WithType(string type) + { + _type = type; + return this; + } + + public ServiceProviderConfigurationBuilder WithToken(string token) + { + _token = token; + return this; + } + + public ServiceProviderConfigurationBuilder WithConfigurationKey(string configurationKey) + { + _configurationKey = configurationKey; + return this; + } + + public ServiceProviderConfiguration Build() + { + return new ServiceProviderConfiguration(_type, _serviceDiscoveryProviderHost, _serviceDiscoveryProviderPort, _token, _configurationKey); + } + } +} diff --git a/src/Ocelot/Configuration/Creator/ServiceProviderConfigurationCreator.cs b/src/Ocelot/Configuration/Creator/ServiceProviderConfigurationCreator.cs index aa735f1a..7fbb49e4 100644 --- a/src/Ocelot/Configuration/Creator/ServiceProviderConfigurationCreator.cs +++ b/src/Ocelot/Configuration/Creator/ServiceProviderConfigurationCreator.cs @@ -1,21 +1,22 @@ -using Ocelot.Configuration.Builder; -using Ocelot.Configuration.File; - -namespace Ocelot.Configuration.Creator -{ - public class ServiceProviderConfigurationCreator : IServiceProviderConfigurationCreator - { - public ServiceProviderConfiguration Create(FileGlobalConfiguration globalConfiguration) - { - //todo log or return error here dont just default to something that wont work.. - var serviceProviderPort = globalConfiguration?.ServiceDiscoveryProvider?.Port ?? 0; - - return new ServiceProviderConfigurationBuilder() - .WithHost(globalConfiguration?.ServiceDiscoveryProvider?.Host) - .WithPort(serviceProviderPort) - .WithType(globalConfiguration?.ServiceDiscoveryProvider?.Type) - .WithToken(globalConfiguration?.ServiceDiscoveryProvider?.Token) - .Build(); - } - } -} +using Ocelot.Configuration.Builder; +using Ocelot.Configuration.File; + +namespace Ocelot.Configuration.Creator +{ + public class ServiceProviderConfigurationCreator : IServiceProviderConfigurationCreator + { + public ServiceProviderConfiguration Create(FileGlobalConfiguration globalConfiguration) + { + //todo log or return error here dont just default to something that wont work.. + var serviceProviderPort = globalConfiguration?.ServiceDiscoveryProvider?.Port ?? 0; + + return new ServiceProviderConfigurationBuilder() + .WithHost(globalConfiguration?.ServiceDiscoveryProvider?.Host) + .WithPort(serviceProviderPort) + .WithType(globalConfiguration?.ServiceDiscoveryProvider?.Type) + .WithToken(globalConfiguration?.ServiceDiscoveryProvider?.Token) + .WithConfigurationKey(globalConfiguration?.ServiceDiscoveryProvider?.ConfigurationKey) + .Build(); + } + } +} diff --git a/src/Ocelot/Configuration/File/FileServiceDiscoveryProvider.cs b/src/Ocelot/Configuration/File/FileServiceDiscoveryProvider.cs index 9a96a6d3..df6c7e27 100644 --- a/src/Ocelot/Configuration/File/FileServiceDiscoveryProvider.cs +++ b/src/Ocelot/Configuration/File/FileServiceDiscoveryProvider.cs @@ -1,10 +1,11 @@ -namespace Ocelot.Configuration.File -{ - public class FileServiceDiscoveryProvider - { - public string Host {get;set;} - public int Port { get; set; } - public string Type { get; set; } - public string Token { get; set; } - } -} +namespace Ocelot.Configuration.File +{ + public class FileServiceDiscoveryProvider + { + public string Host {get;set;} + public int Port { get; set; } + public string Type { get; set; } + public string Token { get; set; } + public string ConfigurationKey { get; set; } + } +} diff --git a/src/Ocelot/Configuration/Repository/ConsulFileConfigurationRepository.cs b/src/Ocelot/Configuration/Repository/ConsulFileConfigurationRepository.cs index 165e035b..e1b6c086 100644 --- a/src/Ocelot/Configuration/Repository/ConsulFileConfigurationRepository.cs +++ b/src/Ocelot/Configuration/Repository/ConsulFileConfigurationRepository.cs @@ -1,96 +1,99 @@ -namespace Ocelot.Configuration.Repository -{ - using System; - using System.Text; - using System.Threading.Tasks; - using Consul; - using Newtonsoft.Json; - using Ocelot.Configuration.File; - using Ocelot.Infrastructure.Consul; - using Ocelot.Logging; - using Ocelot.Responses; - using Ocelot.ServiceDiscovery.Configuration; - - public class ConsulFileConfigurationRepository : IFileConfigurationRepository - { - private readonly IConsulClient _consul; - private const string OcelotConfiguration = "InternalConfiguration"; - private readonly Cache.IOcelotCache _cache; - private readonly IOcelotLogger _logger; - - public ConsulFileConfigurationRepository( - Cache.IOcelotCache cache, - IInternalConfigurationRepository repo, - IConsulClientFactory factory, - IOcelotLoggerFactory loggerFactory) - { - _logger = loggerFactory.CreateLogger(); - _cache = cache; - - var internalConfig = repo.Get(); - - var consulHost = "localhost"; - var consulPort = 8500; - string token = null; - - if (!internalConfig.IsError) - { - consulHost = string.IsNullOrEmpty(internalConfig.Data.ServiceProviderConfiguration?.Host) ? consulHost : internalConfig.Data.ServiceProviderConfiguration?.Host; - consulPort = internalConfig.Data.ServiceProviderConfiguration?.Port ?? consulPort; - token = internalConfig.Data.ServiceProviderConfiguration?.Token; - } - - var config = new ConsulRegistryConfiguration(consulHost, consulPort, OcelotConfiguration, token); - - _consul = factory.Get(config); - } - - public async Task> Get() - { - var config = _cache.Get(OcelotConfiguration, OcelotConfiguration); - - if (config != null) - { - return new OkResponse(config); - } - - var queryResult = await _consul.KV.Get(OcelotConfiguration); - - if (queryResult.Response == null) - { - return new OkResponse(null); - } - - var bytes = queryResult.Response.Value; - - var json = Encoding.UTF8.GetString(bytes); - - var consulConfig = JsonConvert.DeserializeObject(json); - - return new OkResponse(consulConfig); - } - - public async Task Set(FileConfiguration ocelotConfiguration) - { - var json = JsonConvert.SerializeObject(ocelotConfiguration, Formatting.Indented); - - var bytes = Encoding.UTF8.GetBytes(json); - - var kvPair = new KVPair(OcelotConfiguration) - { - Value = bytes - }; - - var result = await _consul.KV.Put(kvPair); - - if (result.Response) - { - _cache.AddAndDelete(OcelotConfiguration, ocelotConfiguration, TimeSpan.FromSeconds(3), OcelotConfiguration); - - return new OkResponse(); - } - - return new ErrorResponse(new UnableToSetConfigInConsulError($"Unable to set FileConfiguration in consul, response status code from consul was {result.StatusCode}")); - } - } -} +namespace Ocelot.Configuration.Repository +{ + using System; + using System.Text; + using System.Threading.Tasks; + using Consul; + using Newtonsoft.Json; + using Ocelot.Configuration.File; + using Ocelot.Infrastructure.Consul; + using Ocelot.Logging; + using Ocelot.Responses; + using Ocelot.ServiceDiscovery.Configuration; + + public class ConsulFileConfigurationRepository : IFileConfigurationRepository + { + private readonly IConsulClient _consul; + private readonly string _configurationKey; + private readonly Cache.IOcelotCache _cache; + private readonly IOcelotLogger _logger; + + public ConsulFileConfigurationRepository( + Cache.IOcelotCache cache, + IInternalConfigurationRepository repo, + IConsulClientFactory factory, + IOcelotLoggerFactory loggerFactory) + { + _logger = loggerFactory.CreateLogger(); + _cache = cache; + + var internalConfig = repo.Get(); + + _configurationKey = "InternalConfiguration"; + var consulHost = "localhost"; + var consulPort = 8500; + string token = null; + + if (!internalConfig.IsError) + { + consulHost = string.IsNullOrEmpty(internalConfig.Data.ServiceProviderConfiguration?.Host) ? consulHost : internalConfig.Data.ServiceProviderConfiguration?.Host; + consulPort = internalConfig.Data.ServiceProviderConfiguration?.Port ?? consulPort; + token = internalConfig.Data.ServiceProviderConfiguration?.Token; + _configurationKey = !string.IsNullOrEmpty(internalConfig.Data.ServiceProviderConfiguration?.ConfigurationKey) ? + internalConfig.Data.ServiceProviderConfiguration?.ConfigurationKey : _configurationKey; + } + + var config = new ConsulRegistryConfiguration(consulHost, consulPort, _configurationKey, token); + + _consul = factory.Get(config); + } + + public async Task> Get() + { + var config = _cache.Get(_configurationKey, _configurationKey); + + if (config != null) + { + return new OkResponse(config); + } + + var queryResult = await _consul.KV.Get(_configurationKey); + + if (queryResult.Response == null) + { + return new OkResponse(null); + } + + var bytes = queryResult.Response.Value; + + var json = Encoding.UTF8.GetString(bytes); + + var consulConfig = JsonConvert.DeserializeObject(json); + + return new OkResponse(consulConfig); + } + + public async Task Set(FileConfiguration ocelotConfiguration) + { + var json = JsonConvert.SerializeObject(ocelotConfiguration, Formatting.Indented); + + var bytes = Encoding.UTF8.GetBytes(json); + + var kvPair = new KVPair(_configurationKey) + { + Value = bytes + }; + + var result = await _consul.KV.Put(kvPair); + + if (result.Response) + { + _cache.AddAndDelete(_configurationKey, ocelotConfiguration, TimeSpan.FromSeconds(3), _configurationKey); + + return new OkResponse(); + } + + return new ErrorResponse(new UnableToSetConfigInConsulError($"Unable to set FileConfiguration in consul, response status code from consul was {result.StatusCode}")); + } + } +} diff --git a/src/Ocelot/Configuration/ServiceProviderConfiguration.cs b/src/Ocelot/Configuration/ServiceProviderConfiguration.cs index 129d24db..c0bb30fb 100644 --- a/src/Ocelot/Configuration/ServiceProviderConfiguration.cs +++ b/src/Ocelot/Configuration/ServiceProviderConfiguration.cs @@ -2,8 +2,9 @@ { public class ServiceProviderConfiguration { - public ServiceProviderConfiguration(string type, string host, int port, string token) + public ServiceProviderConfiguration(string type, string host, int port, string token, string configurationKey) { + ConfigurationKey = configurationKey; Host = host; Port = port; Token = token; @@ -14,5 +15,6 @@ public int Port { get; } public string Type { get; } public string Token { get; } + public string ConfigurationKey { get; } } } diff --git a/test/Ocelot.UnitTests/Configuration/ConsulFileConfigurationRepositoryTests.cs b/test/Ocelot.UnitTests/Configuration/ConsulFileConfigurationRepositoryTests.cs index 784b38d4..6642bb8d 100644 --- a/test/Ocelot.UnitTests/Configuration/ConsulFileConfigurationRepositoryTests.cs +++ b/test/Ocelot.UnitTests/Configuration/ConsulFileConfigurationRepositoryTests.cs @@ -1,219 +1,260 @@ -namespace Ocelot.UnitTests.Configuration -{ - using Xunit; - using TestStack.BDDfy; - using Shouldly; - using Ocelot.Configuration.Repository; - using Moq; - using Ocelot.Infrastructure.Consul; - using Ocelot.Logging; - using Ocelot.Configuration.File; - using Ocelot.Cache; - using System; - using System.Collections.Generic; - using Ocelot.Responses; - using System.Threading.Tasks; - using Ocelot.Configuration; - using Ocelot.Configuration.Builder; - using Ocelot.ServiceDiscovery.Configuration; - using Consul; - using Newtonsoft.Json; - using System.Text; - using System.Threading; - using System.Linq; - - public class ConsulFileConfigurationRepositoryTests - { - private ConsulFileConfigurationRepository _repo; - private Mock> _cache; - private Mock _internalRepo; - private Mock _factory; - private Mock _loggerFactory; - private Mock _client; - private Mock _kvEndpoint; - private FileConfiguration _fileConfiguration; - private Response _setResult; - private Response _getResult; - - public ConsulFileConfigurationRepositoryTests() - { - _cache = new Mock>(); - _internalRepo = new Mock(); - _loggerFactory = new Mock(); - - _factory = new Mock(); - _client = new Mock(); - _kvEndpoint = new Mock(); - - _client - .Setup(x => x.KV) - .Returns(_kvEndpoint.Object); - _factory - .Setup(x => x.Get(It.IsAny())) - .Returns(_client.Object); - - _internalRepo - .Setup(x => x.Get()) - .Returns(new OkResponse(new InternalConfiguration(new List(), "", new ServiceProviderConfigurationBuilder().Build(), ""))); - - _repo = new ConsulFileConfigurationRepository(_cache.Object, _internalRepo.Object, _factory.Object, _loggerFactory.Object); - } - - [Fact] - public void should_set_config() - { - var config = FakeFileConfiguration(); - - this.Given(_ => GivenIHaveAConfiguration(config)) - .And(_ => GivenWritingToConsulSucceeds()) - .When(_ => WhenISetTheConfiguration()) - .Then(_ => ThenTheConfigurationIsStoredAs(config)) - .BDDfy(); - } - - [Fact] - public void should_get_config() - { - var config = FakeFileConfiguration(); - - this.Given(_ => GivenIHaveAConfiguration(config)) - .And(_ => GivenFetchFromConsulSucceeds()) - .When(_ => WhenIGetTheConfiguration()) - .Then(_ => ThenTheConfigurationIs(config)) - .BDDfy(); - } - - [Fact] - public void should_get_null_config() - { - this.Given(_ => GivenFetchFromConsulReturnsNull()) - .When(_ => WhenIGetTheConfiguration()) - .Then(_ => ThenTheConfigurationIsNull()) - .BDDfy(); - } - - [Fact] - public void should_get_config_from_cache() - { - var config = FakeFileConfiguration(); - - this.Given(_ => GivenIHaveAConfiguration(config)) - .And(_ => GivenFetchFromCacheSucceeds()) - .When(_ => WhenIGetTheConfiguration()) - .Then(_ => ThenTheConfigurationIs(config)) - .BDDfy(); - } - - private void ThenTheConfigurationIsNull() - { - _getResult.Data.ShouldBeNull(); - } - - private void ThenTheConfigurationIs(FileConfiguration config) - { - var expected = JsonConvert.SerializeObject(config, Formatting.Indented); - var result = JsonConvert.SerializeObject(_getResult.Data, Formatting.Indented); - result.ShouldBe(expected); - } - - private async Task WhenIGetTheConfiguration() - { - _getResult = await _repo.Get(); - } - - private void GivenWritingToConsulSucceeds() - { - var response = new WriteResult(); - response.Response = true; - - _kvEndpoint - .Setup(x => x.Put(It.IsAny(), It.IsAny())).ReturnsAsync(response); - } - - private void GivenFetchFromCacheSucceeds() - { - _cache.Setup(x => x.Get(It.IsAny(), It.IsAny())).Returns(_fileConfiguration); - } - - private void GivenFetchFromConsulReturnsNull() - { - QueryResult result = new QueryResult(); - - _kvEndpoint - .Setup(x => x.Get(It.IsAny(), It.IsAny())) - .ReturnsAsync(result); - } - - private void GivenFetchFromConsulSucceeds() - { - var json = JsonConvert.SerializeObject(_fileConfiguration, Formatting.Indented); - - var bytes = Encoding.UTF8.GetBytes(json); - - var kvp = new KVPair("OcelotConfiguration"); - kvp.Value = bytes; - - var query = new QueryResult(); - query.Response = kvp; - - _kvEndpoint - .Setup(x => x.Get(It.IsAny(), It.IsAny())) - .ReturnsAsync(query); - } - - private void ThenTheConfigurationIsStoredAs(FileConfiguration config) - { - var json = JsonConvert.SerializeObject(config, Formatting.Indented); - - var bytes = Encoding.UTF8.GetBytes(json); - - _kvEndpoint - .Verify(x => x.Put(It.Is(k => k.Value.SequenceEqual(bytes)), It.IsAny()), Times.Once); - } - - private async Task WhenISetTheConfiguration() - { - _setResult = await _repo.Set(_fileConfiguration); - } - - private void GivenIHaveAConfiguration(FileConfiguration config) - { - _fileConfiguration = config; - } - - private FileConfiguration FakeFileConfiguration() - { - var reRoutes = new List - { - new FileReRoute - { - DownstreamHostAndPorts = new List - { - new FileHostAndPort - { - Host = "123.12.12.12", - Port = 80, - } - }, - DownstreamScheme = "https", - DownstreamPathTemplate = "/asdfs/test/{test}" - } - }; - - var globalConfiguration = new FileGlobalConfiguration - { - ServiceDiscoveryProvider = new FileServiceDiscoveryProvider - { - Port = 198, - Host = "blah" - } - }; - - return new FileConfiguration - { - GlobalConfiguration = globalConfiguration, - ReRoutes = reRoutes - }; - } - } -} +namespace Ocelot.UnitTests.Configuration +{ + using Xunit; + using TestStack.BDDfy; + using Shouldly; + using Ocelot.Configuration.Repository; + using Moq; + using Ocelot.Infrastructure.Consul; + using Ocelot.Logging; + using Ocelot.Configuration.File; + using Ocelot.Cache; + using System; + using System.Collections.Generic; + using Ocelot.Responses; + using System.Threading.Tasks; + using Ocelot.Configuration; + using Ocelot.Configuration.Builder; + using Ocelot.ServiceDiscovery.Configuration; + using Consul; + using Newtonsoft.Json; + using System.Text; + using System.Threading; + using System.Linq; + + public class ConsulFileConfigurationRepositoryTests + { + private ConsulFileConfigurationRepository _repo; + private Mock> _cache; + private Mock _internalRepo; + private Mock _factory; + private Mock _loggerFactory; + private Mock _client; + private Mock _kvEndpoint; + private FileConfiguration _fileConfiguration; + private Response _setResult; + private Response _getResult; + + public ConsulFileConfigurationRepositoryTests() + { + _cache = new Mock>(); + _internalRepo = new Mock(); + _loggerFactory = new Mock(); + + _factory = new Mock(); + _client = new Mock(); + _kvEndpoint = new Mock(); + + _client + .Setup(x => x.KV) + .Returns(_kvEndpoint.Object); + + _factory + .Setup(x => x.Get(It.IsAny())) + .Returns(_client.Object); + + _internalRepo + .Setup(x => x.Get()) + .Returns(new OkResponse(new InternalConfiguration(new List(), "", new ServiceProviderConfigurationBuilder().Build(), ""))); + + _repo = new ConsulFileConfigurationRepository(_cache.Object, _internalRepo.Object, _factory.Object, _loggerFactory.Object); + } + + [Fact] + public void should_set_config() + { + var config = FakeFileConfiguration(); + + this.Given(_ => GivenIHaveAConfiguration(config)) + .And(_ => GivenWritingToConsulSucceeds()) + .When(_ => WhenISetTheConfiguration()) + .Then(_ => ThenTheConfigurationIsStoredAs(config)) + .BDDfy(); + } + + [Fact] + public void should_get_config() + { + var config = FakeFileConfiguration(); + + this.Given(_ => GivenIHaveAConfiguration(config)) + .And(_ => GivenFetchFromConsulSucceeds()) + .When(_ => WhenIGetTheConfiguration()) + .Then(_ => ThenTheConfigurationIs(config)) + .BDDfy(); + } + + [Fact] + public void should_get_null_config() + { + this.Given(_ => GivenFetchFromConsulReturnsNull()) + .When(_ => WhenIGetTheConfiguration()) + .Then(_ => ThenTheConfigurationIsNull()) + .BDDfy(); + } + + [Fact] + public void should_get_config_from_cache() + { + var config = FakeFileConfiguration(); + + this.Given(_ => GivenIHaveAConfiguration(config)) + .And(_ => GivenFetchFromCacheSucceeds()) + .When(_ => WhenIGetTheConfiguration()) + .Then(_ => ThenTheConfigurationIs(config)) + .BDDfy(); + } + + [Fact] + public void should_set_config_key() + { + var config = FakeFileConfiguration(); + + this.Given(_ => GivenIHaveAConfiguration(config)) + .And(_ => GivenTheConfigKeyComesFromFileConfig("Tom")) + .And(_ => GivenFetchFromConsulSucceeds()) + .When(_ => WhenIGetTheConfiguration()) + .And(_ => ThenTheConfigKeyIs("Tom")) + .BDDfy(); + } + + [Fact] + public void should_set_default_config_key() + { + var config = FakeFileConfiguration(); + + this.Given(_ => GivenIHaveAConfiguration(config)) + .And(_ => GivenFetchFromConsulSucceeds()) + .When(_ => WhenIGetTheConfiguration()) + .And(_ => ThenTheConfigKeyIs("InternalConfiguration")) + .BDDfy(); + } + + private void ThenTheConfigKeyIs(string expected) + { + _kvEndpoint + .Verify(x => x.Get(expected, It.IsAny()), Times.Once); + } + + private void GivenTheConfigKeyComesFromFileConfig(string key) + { + _internalRepo + .Setup(x => x.Get()) + .Returns(new OkResponse(new InternalConfiguration(new List(), "", new ServiceProviderConfigurationBuilder().WithConfigurationKey(key).Build(), ""))); + + _repo = new ConsulFileConfigurationRepository(_cache.Object, _internalRepo.Object, _factory.Object, _loggerFactory.Object); + } + + private void ThenTheConfigurationIsNull() + { + _getResult.Data.ShouldBeNull(); + } + + private void ThenTheConfigurationIs(FileConfiguration config) + { + var expected = JsonConvert.SerializeObject(config, Formatting.Indented); + var result = JsonConvert.SerializeObject(_getResult.Data, Formatting.Indented); + result.ShouldBe(expected); + } + + private async Task WhenIGetTheConfiguration() + { + _getResult = await _repo.Get(); + } + + private void GivenWritingToConsulSucceeds() + { + var response = new WriteResult(); + response.Response = true; + + _kvEndpoint + .Setup(x => x.Put(It.IsAny(), It.IsAny())).ReturnsAsync(response); + } + + private void GivenFetchFromCacheSucceeds() + { + _cache.Setup(x => x.Get(It.IsAny(), It.IsAny())).Returns(_fileConfiguration); + } + + private void GivenFetchFromConsulReturnsNull() + { + QueryResult result = new QueryResult(); + + _kvEndpoint + .Setup(x => x.Get(It.IsAny(), It.IsAny())) + .ReturnsAsync(result); + } + + private void GivenFetchFromConsulSucceeds() + { + var json = JsonConvert.SerializeObject(_fileConfiguration, Formatting.Indented); + + var bytes = Encoding.UTF8.GetBytes(json); + + var kvp = new KVPair("OcelotConfiguration"); + kvp.Value = bytes; + + var query = new QueryResult(); + query.Response = kvp; + + _kvEndpoint + .Setup(x => x.Get(It.IsAny(), It.IsAny())) + .ReturnsAsync(query); + } + + private void ThenTheConfigurationIsStoredAs(FileConfiguration config) + { + var json = JsonConvert.SerializeObject(config, Formatting.Indented); + + var bytes = Encoding.UTF8.GetBytes(json); + + _kvEndpoint + .Verify(x => x.Put(It.Is(k => k.Value.SequenceEqual(bytes)), It.IsAny()), Times.Once); + } + + private async Task WhenISetTheConfiguration() + { + _setResult = await _repo.Set(_fileConfiguration); + } + + private void GivenIHaveAConfiguration(FileConfiguration config) + { + _fileConfiguration = config; + } + + private FileConfiguration FakeFileConfiguration() + { + var reRoutes = new List + { + new FileReRoute + { + DownstreamHostAndPorts = new List + { + new FileHostAndPort + { + Host = "123.12.12.12", + Port = 80, + } + }, + DownstreamScheme = "https", + DownstreamPathTemplate = "/asdfs/test/{test}" + } + }; + + var globalConfiguration = new FileGlobalConfiguration + { + ServiceDiscoveryProvider = new FileServiceDiscoveryProvider + { + Port = 198, + Host = "blah" + } + }; + + return new FileConfiguration + { + GlobalConfiguration = globalConfiguration, + ReRoutes = reRoutes + }; + } + } +} diff --git a/test/Ocelot.UnitTests/Configuration/ServiceProviderCreatorTests.cs b/test/Ocelot.UnitTests/Configuration/ServiceProviderCreatorTests.cs index 214d6d54..f1ec3a07 100644 --- a/test/Ocelot.UnitTests/Configuration/ServiceProviderCreatorTests.cs +++ b/test/Ocelot.UnitTests/Configuration/ServiceProviderCreatorTests.cs @@ -1,67 +1,70 @@ -using Ocelot.Configuration; -using Ocelot.Configuration.Builder; -using Ocelot.Configuration.Creator; -using Ocelot.Configuration.File; -using Shouldly; -using TestStack.BDDfy; -using Xunit; - -namespace Ocelot.UnitTests.Configuration -{ - public class ServiceProviderCreatorTests - { - private readonly ServiceProviderConfigurationCreator _creator; - private FileGlobalConfiguration _globalConfig; - private ServiceProviderConfiguration _result; - - public ServiceProviderCreatorTests() - { - _creator = new ServiceProviderConfigurationCreator(); - } - - [Fact] - public void should_create_service_provider_config() - { - var globalConfig = new FileGlobalConfiguration - { - ServiceDiscoveryProvider = new FileServiceDiscoveryProvider - { - Host = "127.0.0.1", - Port = 1234, - Type = "ServiceFabric", - Token = "testtoken" - } - }; - - var expected = new ServiceProviderConfigurationBuilder() - .WithHost("127.0.0.1") - .WithPort(1234) - .WithType("ServiceFabric") - .WithToken("testtoken") - .Build(); - - this.Given(x => x.GivenTheFollowingGlobalConfig(globalConfig)) - .When(x => x.WhenICreate()) - .Then(x => x.ThenTheConfigIs(expected)) - .BDDfy(); - } - - private void GivenTheFollowingGlobalConfig(FileGlobalConfiguration fileGlobalConfig) - { - _globalConfig = fileGlobalConfig; - } - - private void WhenICreate() - { - _result = _creator.Create(_globalConfig); - } - - private void ThenTheConfigIs(ServiceProviderConfiguration expected) - { - _result.Host.ShouldBe(expected.Host); - _result.Port.ShouldBe(expected.Port); - _result.Token.ShouldBe(expected.Token); - _result.Type.ShouldBe(expected.Type); - } - } -} +using Ocelot.Configuration; +using Ocelot.Configuration.Builder; +using Ocelot.Configuration.Creator; +using Ocelot.Configuration.File; +using Shouldly; +using TestStack.BDDfy; +using Xunit; + +namespace Ocelot.UnitTests.Configuration +{ + public class ServiceProviderCreatorTests + { + private readonly ServiceProviderConfigurationCreator _creator; + private FileGlobalConfiguration _globalConfig; + private ServiceProviderConfiguration _result; + + public ServiceProviderCreatorTests() + { + _creator = new ServiceProviderConfigurationCreator(); + } + + [Fact] + public void should_create_service_provider_config() + { + var globalConfig = new FileGlobalConfiguration + { + ServiceDiscoveryProvider = new FileServiceDiscoveryProvider + { + Host = "127.0.0.1", + Port = 1234, + Type = "ServiceFabric", + Token = "testtoken", + ConfigurationKey = "woo" + } + }; + + var expected = new ServiceProviderConfigurationBuilder() + .WithHost("127.0.0.1") + .WithPort(1234) + .WithType("ServiceFabric") + .WithToken("testtoken") + .WithConfigurationKey("woo") + .Build(); + + this.Given(x => x.GivenTheFollowingGlobalConfig(globalConfig)) + .When(x => x.WhenICreate()) + .Then(x => x.ThenTheConfigIs(expected)) + .BDDfy(); + } + + private void GivenTheFollowingGlobalConfig(FileGlobalConfiguration fileGlobalConfig) + { + _globalConfig = fileGlobalConfig; + } + + private void WhenICreate() + { + _result = _creator.Create(_globalConfig); + } + + private void ThenTheConfigIs(ServiceProviderConfiguration expected) + { + _result.Host.ShouldBe(expected.Host); + _result.Port.ShouldBe(expected.Port); + _result.Token.ShouldBe(expected.Token); + _result.Type.ShouldBe(expected.Type); + _result.ConfigurationKey.ShouldBe(expected.ConfigurationKey); + } + } +} diff --git a/test/Ocelot.UnitTests/LoadBalancer/LoadBalancerHouseTests.cs b/test/Ocelot.UnitTests/LoadBalancer/LoadBalancerHouseTests.cs index ffb84518..1ea409df 100644 --- a/test/Ocelot.UnitTests/LoadBalancer/LoadBalancerHouseTests.cs +++ b/test/Ocelot.UnitTests/LoadBalancer/LoadBalancerHouseTests.cs @@ -26,7 +26,7 @@ namespace Ocelot.UnitTests.LoadBalancer { _factory = new Mock(); _loadBalancerHouse = new LoadBalancerHouse(_factory.Object); - _serviceProviderConfig = new ServiceProviderConfiguration("myType","myHost",123, string.Empty); + _serviceProviderConfig = new ServiceProviderConfiguration("myType","myHost",123, string.Empty, "configKey"); } [Fact]