Raft round 2 (#182)

* brought in rafty

* moved raft classes into Ocelot and deleted from int project

* started to set up rafty in Ocelot

* RAFTY INSIDE OCELOT...WOOT

* more work adding rafty...just need to get auth working now

* rudimentary authenticated raft requests working

* asyn await stuff

* hacked rafty into the fileconfigurationcontroller...everything seems to be working roughly but I have a lot of refactoring to do

* updated to latest rafty that doesnt need an id

* hacky but all tests passing

* changed admin area set up to use builder not configuration.json, changed admin area auth to use client credentials

* missing code coverage

* ignore raft sectionf for code coverage

* ignore raft sectionf for code coverage

* back to normal filters

* try exclude attr

* missed these

* moved client secret to builder for authentication and updated docs

* lock to try and fix error accessing identity server created temprsa file on build server

* updated postman scripts and changed Ocelot to not always use type handling as this looked crap when manually accessing the configuration endpoint

* added rafty docs

* changes I missed

* added serialisation code we need for rafty to process commands when they proxy to leader

* moved controllers into their feature slices

build.cake

@@ -133,7 +133,7 @@ Task("RunUnitTests")
 				new OpenCoverSettings()
 				{
 					Register="user",
-					ArgumentCustomization=args=>args.Append(@"-oldstyle -returntargetcode")
+					ArgumentCustomization=args=>args.Append(@"-oldstyle -returntargetcode -excludebyattribute:*.ExcludeFromCoverage*")
 				}
 				.WithFilter("+[Ocelot*]*")
 				.WithFilter("-[xunit*]*")

docs/features/administration.rst

@@ -6,33 +6,24 @@ using bearer tokens that you request from Ocelot iteself. This is provided by th
 `Identity Server <https://github.com/IdentityServer/IdentityServer4>`_ project that I have been using for a few years now. Check them out.
 
 In order to enable the administration section you need to do a few things. First of all add this to your
-initial configuration.json. The value can be anything you want and it is obviously reccomended don't use
+initial Startup.cs. 
+
+The path can be anything you want and it is obviously reccomended don't use
 a url you would like to route through with Ocelot as this will not work. The administration uses the
 MapWhen functionality of asp.net core and all requests to {root}/administration will be sent there not 
 to the Ocelot middleware.
 
-.. code-block:: json
+The secret is the client secret that Ocelot's internal IdentityServer will use to authenticate requests to the administration API. This can be whatever you want it to be!
 
-    "GlobalConfiguration": {
+.. code-block:: csharp
-        "AdministrationPath": "/administration"
+
+    public virtual void ConfigureServices(IServiceCollection services)
+    {
+        services
+            .AddOcelot(Configuration)
+            .AddAdministration("/administration", "secret");
     }
 
-This will get the admin area set up but not the authentication. 
-Please note that this is a very basic approach to 
-this problem and if needed we can obviously improve on this!
-
-You need to set 3 environmental variables. 
-
-    ``OCELOT_USERNAME``
-
-        This need to be the admin username you want to use with Ocelot.
-    ``OCELOT_HASH``
-    ``OCELOT_SALT``
-        The hash and salt of the password you want to use given hashing algorythm. When requesting bearer tokens for use with the administration api you will need to supply username and password. In order to create a hash and salt of your password please check out HashCreationTests.should_create_hash_and_salt() this technique is based on [this](https://docs.microsoft.com/en-us/aspnet/core/security/data-protection/consumer-apis/password-hashing)
-        using SHA256 rather than SHA1.
-
-
-
 Now if you went with the configuration options above and want to access the API you can use the postman scripts
 called ocelot.postman_collection.json in the solution to change the Ocelot configuration. Obviously these 
 will need to be changed if you are running Ocelot on a different url to http://localhost:5000.
@@ -40,7 +31,6 @@ will need to be changed if you are running Ocelot on a different url to http://l
 The scripts show you how to request a bearer token from ocelot and then use it to GET the existing configuration and POST 
 a configuration.
 
-
 Administration running multiple Ocelot's
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 If you are running multiple Ocelot's in a cluster then you need to use a certificate to sign the bearer tokens used to access the administration API.
@@ -59,21 +49,17 @@ Administration API
 
 **POST {adminPath}/connect/token**
 
-This gets a token for use with the admin area using the username and password we talk about setting above. Under the hood this calls into an IdentityServer hosted within Ocelot.
+This gets a token for use with the admin area using the client credentials we talk about setting above. Under the hood this calls into an IdentityServer hosted within Ocelot.
 
 The body of the request is form-data as follows
 
 ``client_id`` set as admin
 
-``client_secret`` set as secret
+``client_secret`` set as whatever you used when setting up the administration services.
 
 ``scope`` set as admin
 
-``username`` set as whatever you used
+``grant_type`` set as client_credentials
-
-``password`` set aswhatever you used
-
-``grant_type`` set as password
 
 **GET {adminPath}/configuration**
 

docs/features/raft.rst

@@ -0,0 +1,45 @@
+Raft (EXPERIMENTAL DO NOT USE IN PRODUCTION)
+============================================
+
+Ocelot has recenely integrated `Rafty <https://github.com/TomPallister/Rafty>`_ which is an implementation of Raft that I have also been working on over the last year. This project is very experimental so please do not use this feature of Ocelot in production until I think it's OK.
+
+Raft is a distributed concensus algorythm that allows a cluster of servers (Ocelots) to maintain local state without having a centralised database for storing state (e.g. SQL Server). 
+
+In order to enable Rafty in Ocelot you must make the following changes to your Startup.cs.
+
+.. code-block:: csharp
+
+    public virtual void ConfigureServices(IServiceCollection services)
+    {
+         services
+            .AddOcelot(Configuration)
+            .AddAdministration("/administration", "secret")
+            .AddRafty();
+    }
+
+In addition to this you must add a file called peers.json to your main project and it will look as follows
+
+.. code-block:: json
+
+    {
+        "Peers": [{
+                "HostAndPort": "http://localhost:5000"
+            },
+            {
+                "HostAndPort": "http://localhost:5002"
+            },
+            {
+                "HostAndPort": "http://localhost:5003"
+            },
+            {
+                "HostAndPort": "http://localhost:5004"
+            },
+            {
+                "HostAndPort": "http://localhost:5001"
+            }
+        ]
+    }
+
+Each instance of Ocelot must have it's address in the array so that they can communicate using Rafty.
+
+Once you have made these configuration changes you must deploy and start each instance of Ocelot using the addresses in the peers.json file. The servers should then start communicating with each other! You can test if everything is working by posting a configuration update and checking it has replicated to all servers by getting there configuration.

docs/index.rst

@@ -24,6 +24,7 @@ Thanks for taking a look at the Ocelot documentation. Please use the left hand n
    features/authentication
    features/authorisation
    features/administration
+   features/raft
    features/caching
    features/qualityofservice
    features/claimstransformation 

src/Ocelot/Authentication/BearerToken.cs

@@ -0,0 +1,16 @@
+using Newtonsoft.Json;
+
+namespace Ocelot.Authentication
+{
+    class BearerToken
+    {
+        [JsonProperty("access_token")]
+        public string AccessToken { get; set; }
+
+        [JsonProperty("expires_in")]
+        public int ExpiresIn { get; set; }
+
+        [JsonProperty("token_type")]
+        public string TokenType { get; set; }
+    }
+}

src/Ocelot/Cache/OutputCacheController.cs

@@ -5,7 +5,7 @@ using Microsoft.AspNetCore.Mvc;
 using Ocelot.Cache;
 using Ocelot.Configuration.Provider;
 
-namespace Ocelot.Controllers
+namespace Ocelot.Cache
 {
     [Authorize]
     [Route("outputcache")]

src/Ocelot/Configuration/Authentication/OcelotResourceOwnerPasswordValidator.cs

@@ -1,53 +0,0 @@
-using System;
-using System.Linq;
-using System.Threading.Tasks;
-using IdentityServer4.Models;
-using IdentityServer4.Validation;
-using Ocelot.Configuration.Provider;
-
-namespace Ocelot.Configuration.Authentication
-{
-    public class OcelotResourceOwnerPasswordValidator : IResourceOwnerPasswordValidator
-    {
-        private readonly IHashMatcher _matcher;
-        private readonly IIdentityServerConfiguration _identityServerConfiguration;
-
-        public OcelotResourceOwnerPasswordValidator(IHashMatcher matcher, IIdentityServerConfiguration identityServerConfiguration)
-        {
-            _identityServerConfiguration = identityServerConfiguration;
-            _matcher = matcher;
-        }
-
-        public async Task ValidateAsync(ResourceOwnerPasswordValidationContext context)
-        {
-            try
-            {
-                var user = _identityServerConfiguration.Users.FirstOrDefault(u => u.UserName == context.UserName);
-
-                if(user == null)
-                {
-                    context.Result = new GrantValidationResult(
-                            TokenRequestErrors.InvalidGrant,
-                            "invalid custom credential");
-                }
-                else if(_matcher.Match(context.Password, user.Salt, user.Hash))
-                {
-                    context.Result = new GrantValidationResult(
-                        subject: "admin",
-                        authenticationMethod: "custom");
-                }
-                else
-                {
-                    context.Result = new GrantValidationResult(
-                        TokenRequestErrors.InvalidGrant,
-                        "invalid custom credential");
-                }
-            }
-            catch(Exception ex)
-            {
-                Console.WriteLine(ex);
-            }
-
-        }
-    }
-}

src/Ocelot/Configuration/Creator/FileOcelotConfigurationCreator.cs

@@ -9,6 +9,7 @@ using Ocelot.Configuration.Builder;
 using Ocelot.Configuration.File;
 using Ocelot.Configuration.Parser;
 using Ocelot.Configuration.Validator;
+using Ocelot.DependencyInjection;
 using Ocelot.LoadBalancer;
 using Ocelot.LoadBalancer.LoadBalancers;
 using Ocelot.Logging;
@@ -35,6 +36,8 @@ namespace Ocelot.Configuration.Creator
         private readonly IRateLimitOptionsCreator _rateLimitOptionsCreator;
         private readonly IRegionCreator _regionCreator;
         private readonly IHttpHandlerOptionsCreator _httpHandlerOptionsCreator;
+        private readonly IAdministrationPath _adminPath;
+
 
         public FileOcelotConfigurationCreator(
             IOptions<FileConfiguration> options, 
@@ -49,9 +52,11 @@ namespace Ocelot.Configuration.Creator
             IReRouteOptionsCreator fileReRouteOptionsCreator,
             IRateLimitOptionsCreator rateLimitOptionsCreator,
             IRegionCreator regionCreator,
-            IHttpHandlerOptionsCreator httpHandlerOptionsCreator
+            IHttpHandlerOptionsCreator httpHandlerOptionsCreator,
+            IAdministrationPath adminPath
             )
         {
+            _adminPath = adminPath;
             _regionCreator = regionCreator;
             _rateLimitOptionsCreator = rateLimitOptionsCreator;
             _requestIdKeyCreator = requestIdKeyCreator;
@@ -92,7 +97,7 @@ namespace Ocelot.Configuration.Creator
 
             var serviceProviderConfiguration = _serviceProviderConfigCreator.Create(fileConfiguration.GlobalConfiguration);
             
-            var config = new OcelotConfiguration(reRoutes, fileConfiguration.GlobalConfiguration.AdministrationPath, serviceProviderConfiguration);
+            var config = new OcelotConfiguration(reRoutes, _adminPath.Path, serviceProviderConfiguration);
 
             return new OkResponse<IOcelotConfiguration>(config);
         }

src/Ocelot/Configuration/Creator/IdentityServerConfigurationCreator.cs

@@ -8,29 +8,16 @@ namespace Ocelot.Configuration.Creator
 {
     public static class IdentityServerConfigurationCreator
     {
-        public static IdentityServerConfiguration GetIdentityServerConfiguration()
+        public static IdentityServerConfiguration GetIdentityServerConfiguration(string secret)
         {
-            var username = Environment.GetEnvironmentVariable("OCELOT_USERNAME");
-            var hash = Environment.GetEnvironmentVariable("OCELOT_HASH");
-            var salt = Environment.GetEnvironmentVariable("OCELOT_SALT");
             var credentialsSigningCertificateLocation = Environment.GetEnvironmentVariable("OCELOT_CERTIFICATE");
             var credentialsSigningCertificatePassword = Environment.GetEnvironmentVariable("OCELOT_CERTIFICATE_PASSWORD");
 
             return new IdentityServerConfiguration(
                 "admin",
                 false,
-                SupportedTokens.Both,
+                secret,
-                "secret",
                 new List<string> { "admin", "openid", "offline_access" },
-                "Ocelot Administration",
-                true,
-                GrantTypes.ResourceOwnerPassword,
-                AccessTokenType.Jwt,
-                false,
-                new List<User>
-                {
-                    new User("admin", username, hash, salt)
-                },
                 credentialsSigningCertificateLocation,
                 credentialsSigningCertificatePassword
             );

src/Ocelot/Configuration/File/FileGlobalConfiguration.cs

@@ -12,7 +12,6 @@ namespace Ocelot.Configuration.File
         public string RequestIdKey { get; set; }
 
         public FileServiceDiscoveryProvider ServiceDiscoveryProvider {get;set;}
-        public string AdministrationPath {get;set;}
 
         public FileRateLimitOptions RateLimitOptions { get; set; }
     }

src/Ocelot/Configuration/FileConfigurationController.cs

@@ -1,11 +1,15 @@
+using System;
 using System.Threading.Tasks;
 using Microsoft.AspNetCore.Authorization;
 using Microsoft.AspNetCore.Mvc;
+using Microsoft.Extensions.DependencyInjection;
 using Ocelot.Configuration.File;
 using Ocelot.Configuration.Provider;
 using Ocelot.Configuration.Setter;
+using Ocelot.Raft;
+using Rafty.Concensus;
 
-namespace Ocelot.Controllers
+namespace Ocelot.Configuration
 {
     [Authorize]
     [Route("configuration")]
@@ -13,11 +17,13 @@ namespace Ocelot.Controllers
     {
         private readonly IFileConfigurationProvider _configGetter;
         private readonly IFileConfigurationSetter _configSetter;
+        private readonly IServiceProvider _serviceProvider;
 
-        public FileConfigurationController(IFileConfigurationProvider getFileConfig, IFileConfigurationSetter configSetter)
+        public FileConfigurationController(IFileConfigurationProvider getFileConfig, IFileConfigurationSetter configSetter, IServiceProvider serviceProvider)
         {
             _configGetter = getFileConfig;
             _configSetter = configSetter;
+            _serviceProvider = serviceProvider;
         }
 
         [HttpGet]
@@ -36,9 +42,23 @@ namespace Ocelot.Controllers
         [HttpPost]
         public async Task<IActionResult> Post([FromBody]FileConfiguration fileConfiguration)
         {
+            //todo - this code is a bit shit sort it out..
+            var test = _serviceProvider.GetService(typeof(INode));
+            if (test != null)
+            {
+                var node = (INode)test;
+                var result = node.Accept(new UpdateFileConfiguration(fileConfiguration));
+                if (result.GetType() == typeof(Rafty.Concensus.ErrorResponse<UpdateFileConfiguration>))
+                {
+                    return new BadRequestObjectResult("There was a problem. This error message sucks raise an issue in GitHub.");
+                }
+
+                return new OkObjectResult(result.Command.Configuration);
+            }
+
             var response = await _configSetter.Set(fileConfiguration);
 
-            if(response.IsError)
+            if (response.IsError)
             {
                 return new BadRequestObjectResult(response.Errors);
             }

src/Ocelot/Configuration/Provider/IIdentityServerConfiguration.cs

@@ -7,16 +7,9 @@ namespace Ocelot.Configuration.Provider
     public interface IIdentityServerConfiguration
     {
         string ApiName { get;  }
+        string ApiSecret { get;  }
         bool RequireHttps { get;  }
         List<string> AllowedScopes { get;  }
-        SupportedTokens SupportedTokens { get;  }
-        string ApiSecret { get;  }
-        string Description {get;}
-        bool Enabled {get;}
-        IEnumerable<string>  AllowedGrantTypes {get;}
-        AccessTokenType AccessTokenType {get;}
-        bool RequireClientSecret {get;}
-        List<User> Users {get;}
         string CredentialsSigningCertificateLocation { get; }
         string CredentialsSigningCertificatePassword { get; }
     }

src/Ocelot/Configuration/Provider/IdentityServerConfiguration.cs

@@ -9,27 +9,15 @@ namespace Ocelot.Configuration.Provider
         public IdentityServerConfiguration(
             string apiName, 
             bool requireHttps, 
-            SupportedTokens supportedTokens, 
             string apiSecret,
             List<string> allowedScopes,
-            string description,
+            string credentialsSigningCertificateLocation, 
-            bool enabled,
+            string credentialsSigningCertificatePassword)
-            IEnumerable<string>  grantType,
-            AccessTokenType accessTokenType,
-            bool requireClientSecret,
-            List<User> users, string credentialsSigningCertificateLocation, string credentialsSigningCertificatePassword)
         {
             ApiName = apiName;
             RequireHttps = requireHttps;
-            SupportedTokens = supportedTokens;
             ApiSecret = apiSecret;
             AllowedScopes = allowedScopes;
-            Description = description;
-            Enabled = enabled;
-            AllowedGrantTypes = grantType;
-            AccessTokenType = accessTokenType;
-            RequireClientSecret = requireClientSecret;
-            Users = users;
             CredentialsSigningCertificateLocation = credentialsSigningCertificateLocation;
             CredentialsSigningCertificatePassword = credentialsSigningCertificatePassword;
         }
@@ -37,14 +25,7 @@ namespace Ocelot.Configuration.Provider
         public string ApiName { get; private set; }
         public bool RequireHttps { get; private set; }
         public List<string> AllowedScopes { get; private set; }
-        public SupportedTokens SupportedTokens { get; private set; }
         public string ApiSecret { get; private set; }
-        public string Description {get;private set;}
-        public bool Enabled {get;private set;}
-        public IEnumerable<string>  AllowedGrantTypes {get;private set;}
-        public AccessTokenType AccessTokenType {get;private set;}
-        public bool RequireClientSecret {get;private set;}
-        public List<User> Users {get;private set;}
         public string CredentialsSigningCertificateLocation { get; private set; }
         public string CredentialsSigningCertificatePassword { get; private set; }
     }

src/Ocelot/Configuration/Provider/User.cs

@@ -1,17 +0,0 @@
-namespace Ocelot.Configuration.Provider
-{
-    public class User
-    {
-        public User(string subject, string userName, string hash, string salt)
-        {
-            Subject = subject;
-            UserName = userName;
-            Hash = hash;
-            Salt = salt;
-        }
-        public string Subject { get; private set; }
-        public string UserName { get; private set; }
-        public string Hash { get; private set; }
-        public string Salt { get; private set; }
-    }
-}

src/Ocelot/DependencyInjection/IOcelotBuilder.cs

@@ -7,5 +7,6 @@ namespace Ocelot.DependencyInjection
     {
         IOcelotBuilder AddStoreOcelotConfigurationInConsul();
         IOcelotBuilder AddCacheManager(Action<ConfigurationBuilderCachePart> settings);
+        IOcelotAdministrationBuilder AddAdministration(string path, string secret);
     }
 }

src/Ocelot/DependencyInjection/OcelotBuilder.cs

@@ -14,7 +14,6 @@ using Ocelot.Configuration.Provider;
 using Ocelot.Configuration.Repository;
 using Ocelot.Configuration.Setter;
 using Ocelot.Configuration.Validator;
-using Ocelot.Controllers;
 using Ocelot.DownstreamRouteFinder.Finder;
 using Ocelot.DownstreamRouteFinder.UrlMatcher;
 using Ocelot.DownstreamUrlCreator;
@@ -47,6 +46,12 @@ using Ocelot.Configuration.Builder;
 using FileConfigurationProvider = Ocelot.Configuration.Provider.FileConfigurationProvider;
 using Microsoft.Extensions.DependencyInjection.Extensions;
 using System.Linq;
+using Ocelot.Raft;
+using Rafty.Concensus;
+using Rafty.FiniteStateMachine;
+using Rafty.Infrastructure;
+using Rafty.Log;
+using Newtonsoft.Json;
 
 namespace Ocelot.DependencyInjection
 {
@@ -121,14 +126,6 @@ namespace Ocelot.DependencyInjection
             _services.AddMemoryCache();
             _services.TryAddSingleton<OcelotDiagnosticListener>();
 
-            //add identity server for admin area
-            var identityServerConfiguration = IdentityServerConfigurationCreator.GetIdentityServerConfiguration();
-
-            if (identityServerConfiguration != null)
-            {
-                AddIdentityServer(identityServerConfiguration);
-            }
-
             //add asp.net services..
             var assembly = typeof(FileConfigurationController).GetTypeInfo().Assembly;
 
@@ -141,6 +138,24 @@ namespace Ocelot.DependencyInjection
             _services.AddLogging();
             _services.AddMiddlewareAnalysis();
             _services.AddWebEncoders();
+            _services.AddSingleton<IAdministrationPath>(new NullAdministrationPath());
+        }
+
+        public IOcelotAdministrationBuilder AddAdministration(string path, string secret)
+        {
+            var administrationPath = new AdministrationPath(path);
+
+            //add identity server for admin area
+            var identityServerConfiguration = IdentityServerConfigurationCreator.GetIdentityServerConfiguration(secret);
+
+            if (identityServerConfiguration != null)
+            {
+                AddIdentityServer(identityServerConfiguration, administrationPath);
+            }
+
+            var descriptor = new ServiceDescriptor(typeof(IAdministrationPath), administrationPath);
+            _services.Replace(descriptor);
+            return new OcelotAdministrationBuilder(_services, _configurationRoot);
         }
 
         public IOcelotBuilder AddStoreOcelotConfigurationInConsul()
@@ -185,7 +200,7 @@ namespace Ocelot.DependencyInjection
             return this;
         }
 
-        private void AddIdentityServer(IIdentityServerConfiguration identityServerConfiguration) 
+        private void AddIdentityServer(IIdentityServerConfiguration identityServerConfiguration, IAdministrationPath adminPath) 
         {
             _services.TryAddSingleton<IIdentityServerConfiguration>(identityServerConfiguration);
             _services.TryAddSingleton<IHashMatcher, HashMatcher>();
@@ -194,8 +209,7 @@ namespace Ocelot.DependencyInjection
                     o.IssuerUri = "Ocelot";
                 })
                 .AddInMemoryApiResources(Resources(identityServerConfiguration))
-                .AddInMemoryClients(Client(identityServerConfiguration))
+                .AddInMemoryClients(Client(identityServerConfiguration));
-                .AddResourceOwnerValidator<OcelotResourceOwnerPasswordValidator>();
 
             //todo - refactor a method so we know why this is happening
             var whb = _services.First(x => x.ServiceType == typeof(IWebHostBuilder));
@@ -206,8 +220,7 @@ namespace Ocelot.DependencyInjection
             _services.AddAuthentication(IdentityServerAuthenticationDefaults.AuthenticationScheme)
                 .AddIdentityServerAuthentication(o =>
                 {
-                    var adminPath = _configurationRoot.GetValue("GlobalConfiguration:AdministrationPath", string.Empty);
+                    o.Authority = baseSchemeUrlAndPort + adminPath.Path;
-                    o.Authority = baseSchemeUrlAndPort + adminPath;
                     o.ApiName = identityServerConfiguration.ApiName;
                     o.RequireHttpsMetadata = identityServerConfiguration.RequireHttps;
                     o.SupportedTokens = SupportedTokens.Both;
@@ -240,7 +253,7 @@ namespace Ocelot.DependencyInjection
                             Value = identityServerConfiguration.ApiSecret.Sha256()
                         }
                     }
-                }
+                },
             };
         }
 
@@ -251,12 +264,65 @@ namespace Ocelot.DependencyInjection
                 new Client
                 {
                     ClientId = identityServerConfiguration.ApiName,
-                    AllowedGrantTypes = GrantTypes.ResourceOwnerPassword,
+                    AllowedGrantTypes = GrantTypes.ClientCredentials,
                     ClientSecrets = new List<Secret> {new Secret(identityServerConfiguration.ApiSecret.Sha256())},
                     AllowedScopes = { identityServerConfiguration.ApiName }
                 }
             };
         }
+    }
 
+    public interface IOcelotAdministrationBuilder
+    {
+        IOcelotAdministrationBuilder AddRafty();
+    }
+
+    public class OcelotAdministrationBuilder : IOcelotAdministrationBuilder
+    {
+        private IServiceCollection _services;
+        private IConfigurationRoot _configurationRoot;
+        
+        public OcelotAdministrationBuilder(IServiceCollection services, IConfigurationRoot configurationRoot)
+        {
+            _configurationRoot = configurationRoot;
+            _services = services;    
+        }
+        
+        public IOcelotAdministrationBuilder AddRafty()
+        {
+            var settings = new InMemorySettings(4000, 5000, 100, 5000);
+            _services.AddSingleton<ILog, SqlLiteLog>();
+            _services.AddSingleton<IFiniteStateMachine, OcelotFiniteStateMachine>();
+            _services.AddSingleton<ISettings>(settings);
+            _services.AddSingleton<IPeersProvider, FilePeersProvider>();
+            _services.AddSingleton<INode, Node>();
+            _services.Configure<FilePeers>(_configurationRoot);
+            return this;
+        }
+    }
+
+    public interface IAdministrationPath
+    {
+        string Path {get;}
+    }
+
+    public class NullAdministrationPath : IAdministrationPath
+    {
+        public NullAdministrationPath()
+        {
+            Path = null;
+        }
+
+        public string Path {get;private set;}
+    }
+
+    public class AdministrationPath : IAdministrationPath
+    {
+        public AdministrationPath(string path)
+        {
+            Path = path;
+        }
+
+        public string Path {get;private set;}
     }
 }

src/Ocelot/Middleware/OcelotMiddlewareExtensions.cs

@@ -7,7 +7,6 @@ using Microsoft.Extensions.DependencyInjection;
 using Ocelot.Authentication.Middleware;
 using Ocelot.Cache.Middleware;
 using Ocelot.Claims.Middleware;
-using Ocelot.Controllers;
 using Ocelot.DownstreamRouteFinder.Middleware;
 using Ocelot.DownstreamUrlCreator.Middleware;
 using Ocelot.Errors.Middleware;
@@ -23,12 +22,15 @@ using Ocelot.RateLimit.Middleware;
 namespace Ocelot.Middleware
 {
     using System;
+    using System.IO;
     using System.Linq;
     using System.Threading.Tasks;
     using Authorisation.Middleware;
     using Microsoft.AspNetCore.Hosting;
     using Microsoft.AspNetCore.Http;
+    using Microsoft.Extensions.Logging;
     using Microsoft.Extensions.Options;
+    using Newtonsoft.Json;
     using Ocelot.Configuration;
     using Ocelot.Configuration.Creator;
     using Ocelot.Configuration.File;
@@ -36,7 +38,10 @@ namespace Ocelot.Middleware
     using Ocelot.Configuration.Repository;
     using Ocelot.Configuration.Setter;
     using Ocelot.LoadBalancer.Middleware;
+    using Ocelot.Raft;
     using Ocelot.Responses;
+    using Rafty.Concensus;
+    using Rafty.Infrastructure;
 
     public static class OcelotMiddlewareExtensions
     {
@@ -64,6 +69,11 @@ namespace Ocelot.Middleware
             
             await CreateAdministrationArea(builder, configuration);
 
+            if(UsingRafty(builder))
+            {
+                SetUpRafty(builder);
+            }
+
             ConfigureDiagnosticListener(builder);
 
             // This is registered to catch any global exceptions that are not handled
@@ -149,6 +159,26 @@ namespace Ocelot.Middleware
             return builder;
         }
 
+        private static bool UsingRafty(IApplicationBuilder builder)
+        {
+            var possible = builder.ApplicationServices.GetService(typeof(INode)) as INode;
+            if(possible != null)
+            {
+                return true;
+            }
+
+            return false;
+        }
+
+        private static void SetUpRafty(IApplicationBuilder builder)
+        {
+            var applicationLifetime = (IApplicationLifetime)builder.ApplicationServices.GetService(typeof(IApplicationLifetime));
+            applicationLifetime.ApplicationStopping.Register(() => OnShutdown(builder));
+            var node = (INode)builder.ApplicationServices.GetService(typeof(INode));
+            var nodeId = (NodeId)builder.ApplicationServices.GetService(typeof(NodeId));
+            node.Start(nodeId.Id);
+        }
+
         private static async Task<IOcelotConfiguration> CreateConfiguration(IApplicationBuilder builder)
         {
             var deps = GetDependencies(builder);
@@ -183,7 +213,7 @@ namespace Ocelot.Middleware
             return response == null || response.IsError;
         }
 
-        private static bool ConfigurationNotSetUp(Response<IOcelotConfiguration> ocelotConfiguration)
+        private static bool ConfigurationNotSetUp(Ocelot.Responses.Response<IOcelotConfiguration> ocelotConfiguration)
         {
             return ocelotConfiguration == null || ocelotConfiguration.Data == null || ocelotConfiguration.IsError;
         }
@@ -247,6 +277,7 @@ namespace Ocelot.Middleware
                     return new ErrorResponse(ocelotConfig.Errors);
                 }
                 config = await ocelotConfigurationRepository.AddOrReplace(ocelotConfig.Data);
+                //todo - this starts the poller if it has been registered...please this is so bad.
                 var hack = builder.ApplicationServices.GetService(typeof(ConsulFileConfigurationPoller));
             }
 
@@ -292,5 +323,11 @@ namespace Ocelot.Middleware
                  diagnosticListener.SubscribeWithAdapter(listener);
              }
          }
+        
+        private static void OnShutdown(IApplicationBuilder app)
+        {
+            var node = (INode)app.ApplicationServices.GetService(typeof(INode));
+            node.Stop();
+        }
     }
 }

src/Ocelot/Ocelot.csproj

@@ -1,5 +1,4 @@
-<Project Sdk="Microsoft.NET.Sdk">
+<Project Sdk="Microsoft.NET.Sdk">
-
   <PropertyGroup>
     <TargetFramework>netcoreapp2.0</TargetFramework>
     <RuntimeFrameworkVersion>2.0.0</RuntimeFrameworkVersion>
@@ -11,39 +10,37 @@
     <PackageId>Ocelot</PackageId>
     <PackageTags>API Gateway;.NET core</PackageTags>
     <PackageProjectUrl>https://github.com/TomPallister/Ocelot</PackageProjectUrl>
-	  <PackageProjectUrl>https://github.com/TomPallister/Ocelot</PackageProjectUrl>
+    <PackageProjectUrl>https://github.com/TomPallister/Ocelot</PackageProjectUrl>
     <RuntimeIdentifiers>win10-x64;osx.10.11-x64;osx.10.12-x64;win7-x64</RuntimeIdentifiers>
     <GenerateAssemblyConfigurationAttribute>false</GenerateAssemblyConfigurationAttribute>
     <GenerateAssemblyCompanyAttribute>false</GenerateAssemblyCompanyAttribute>
-	  <GeneratePackageOnBuild>True</GeneratePackageOnBuild>
+    <GeneratePackageOnBuild>True</GeneratePackageOnBuild>
     <GenerateAssemblyProductAttribute>false</GenerateAssemblyProductAttribute>
-	<Authors>Tom Pallister</Authors>
+    <Authors>Tom Pallister</Authors>
   </PropertyGroup>
-
   <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|AnyCPU'">
     <DebugType>full</DebugType>
     <DebugSymbols>True</DebugSymbols>
   </PropertyGroup>
-
   <ItemGroup>
-    <PackageReference Include="FluentValidation" Version="7.2.1" />
+    <PackageReference Include="FluentValidation" Version="7.2.1"/>
-    <PackageReference Include="IdentityServer4.AccessTokenValidation" Version="2.1.0" />
+    <PackageReference Include="IdentityServer4.AccessTokenValidation" Version="2.1.0"/>
-    <PackageReference Include="Microsoft.AspNetCore.All" Version="2.0.0" />
+    <PackageReference Include="Microsoft.AspNetCore.All" Version="2.0.0"/>
-    <PackageReference Include="Microsoft.Extensions.Configuration.EnvironmentVariables" Version="2.0.0" />
+    <PackageReference Include="Microsoft.Extensions.Configuration.EnvironmentVariables" Version="2.0.0"/>
-    <PackageReference Include="Microsoft.Extensions.Configuration.FileExtensions" Version="2.0.0" />
+    <PackageReference Include="Microsoft.Extensions.Configuration.FileExtensions" Version="2.0.0"/>
-    <PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="2.0.0" />
+    <PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="2.0.0"/>
-    <PackageReference Include="Microsoft.Extensions.DiagnosticAdapter" Version="2.0.0" />
+    <PackageReference Include="Microsoft.Extensions.DiagnosticAdapter" Version="2.0.0"/>
-    <PackageReference Include="Microsoft.Extensions.Logging" Version="2.0.0" />
+    <PackageReference Include="Microsoft.Extensions.Logging" Version="2.0.0"/>
-    <PackageReference Include="Microsoft.Extensions.Logging.Console" Version="2.0.0" />
+    <PackageReference Include="Microsoft.Extensions.Logging.Console" Version="2.0.0"/>
-    <PackageReference Include="Microsoft.Extensions.Logging.Debug" Version="2.0.0" />
+    <PackageReference Include="Microsoft.Extensions.Logging.Debug" Version="2.0.0"/>
-    <PackageReference Include="Microsoft.Extensions.Options.ConfigurationExtensions" Version="2.0.0" />
+    <PackageReference Include="Microsoft.Extensions.Options.ConfigurationExtensions" Version="2.0.0"/>
-    <PackageReference Include="System.Text.RegularExpressions" Version="4.3.0" />
+    <PackageReference Include="System.Text.RegularExpressions" Version="4.3.0"/>
-    <PackageReference Include="CacheManager.Core" Version="1.1.1" />
+    <PackageReference Include="CacheManager.Core" Version="1.1.1"/>
-    <PackageReference Include="CacheManager.Microsoft.Extensions.Configuration" Version="1.1.1" />
+    <PackageReference Include="CacheManager.Microsoft.Extensions.Configuration" Version="1.1.1"/>
-    <PackageReference Include="CacheManager.Microsoft.Extensions.Logging" Version="1.1.1" />
+    <PackageReference Include="CacheManager.Microsoft.Extensions.Logging" Version="1.1.1"/>
-    <PackageReference Include="Consul" Version="0.7.2.3" />
+    <PackageReference Include="Consul" Version="0.7.2.3"/>
-    <PackageReference Include="Polly" Version="5.3.1" />
+    <PackageReference Include="Polly" Version="5.3.1"/>
-    <PackageReference Include="IdentityServer4" Version="2.0.2" />
+    <PackageReference Include="IdentityServer4" Version="2.0.2"/>
+    <PackageReference Include="Rafty" Version="0.4.2"/>
   </ItemGroup>
-
 </Project>

src/Ocelot/Raft/ExcludeFromCoverage.cs

@@ -0,0 +1,7 @@
+using System;
+
+namespace Ocelot.Raft
+{
+    [AttributeUsage(AttributeTargets.Class|AttributeTargets.Method|AttributeTargets.Property)]
+    public class ExcludeFromCoverageAttribute : Attribute{}
+}

src/Ocelot/Raft/FakeCommand.cs

@@ -0,0 +1,15 @@
+using Rafty.FiniteStateMachine;
+
+namespace Ocelot.Raft
+{
+    [ExcludeFromCoverage]
+    public class FakeCommand : ICommand
+    {
+        public FakeCommand(string value)
+        {
+            this.Value = value;
+
+        }
+        public string Value { get; private set; }
+    }
+}

src/Ocelot/Raft/FileFsm.cs

@@ -0,0 +1,33 @@
+using System;
+using System.IO;
+using Newtonsoft.Json;
+using Rafty.FiniteStateMachine;
+using Rafty.Infrastructure;
+using Rafty.Log;
+
+namespace Ocelot.Raft
+{    
+    [ExcludeFromCoverage]
+    public class FileFsm : IFiniteStateMachine
+    {
+        private string _id;
+
+        public FileFsm(NodeId nodeId)
+        {
+            _id = nodeId.Id.Replace("/","").Replace(":","");
+        }
+        
+        public void Handle(LogEntry log)
+        {
+            try
+            {
+                var json = JsonConvert.SerializeObject(log.CommandData);
+                File.AppendAllText(_id, json);
+            }
+            catch(Exception exception)
+            {
+                Console.WriteLine(exception);
+            }
+        }
+    }
+}

src/Ocelot/Raft/FilePeer.cs

@@ -0,0 +1,8 @@
+namespace Ocelot.Raft
+{
+    [ExcludeFromCoverage]
+    public class FilePeer
+    {
+        public string HostAndPort { get; set; }
+    }
+}

src/Ocelot/Raft/FilePeers.cs

@@ -0,0 +1,15 @@
+using System.Collections.Generic;
+
+namespace Ocelot.Raft
+{
+    [ExcludeFromCoverage]
+    public class FilePeers
+    {
+        public FilePeers()
+        {
+            Peers = new List<FilePeer>();
+        }
+
+        public List<FilePeer> Peers {get; set;}
+    }
+}

src/Ocelot/Raft/FilePeersProvider.cs

@@ -0,0 +1,44 @@
+using System;
+using System.Collections.Generic;
+using System.Net.Http;
+using Microsoft.AspNetCore.Hosting;
+using Microsoft.Extensions.Options;
+using Ocelot.Configuration;
+using Ocelot.Configuration.Provider;
+using Rafty.Concensus;
+using Rafty.Infrastructure;
+
+namespace Ocelot.Raft
+{
+    [ExcludeFromCoverage]
+    public class FilePeersProvider : IPeersProvider
+    {
+        private readonly IOptions<FilePeers> _options;
+        private List<IPeer> _peers;
+        private IWebHostBuilder _builder;
+        private IOcelotConfigurationProvider _provider;
+        private IIdentityServerConfiguration _identityServerConfig;
+
+        public FilePeersProvider(IOptions<FilePeers> options, IWebHostBuilder builder, IOcelotConfigurationProvider provider, IIdentityServerConfiguration identityServerConfig)
+        {
+            _identityServerConfig = identityServerConfig;
+            _provider = provider;
+            _builder = builder;
+            _options = options;
+            _peers = new List<IPeer>();
+            //todo - sort out async nonsense..
+            var config = _provider.Get().GetAwaiter().GetResult();
+            foreach (var item in _options.Value.Peers)
+            {
+                var httpClient = new HttpClient();
+                //todo what if this errors?
+                var httpPeer = new HttpPeer(item.HostAndPort, httpClient, _builder, config.Data, _identityServerConfig);
+                _peers.Add(httpPeer);
+            }
+        }
+        public List<IPeer> Get()
+        {
+            return _peers;
+        }
+    }
+}

src/Ocelot/Raft/HttpPeer.cs

@@ -0,0 +1,128 @@
+using System;
+using System.Collections.Generic;
+using System.Net.Http;
+using System.Threading.Tasks;
+using Microsoft.AspNetCore.Hosting;
+using Newtonsoft.Json;
+using Ocelot.Authentication;
+using Ocelot.Configuration;
+using Ocelot.Configuration.Provider;
+using Rafty.Concensus;
+using Rafty.FiniteStateMachine;
+
+namespace Ocelot.Raft
+{
+    [ExcludeFromCoverage]
+    public class HttpPeer : IPeer
+    {
+        private string _hostAndPort;
+        private HttpClient _httpClient;
+        private JsonSerializerSettings _jsonSerializerSettings;
+        private string _baseSchemeUrlAndPort;
+        private BearerToken _token;
+        private IOcelotConfiguration _config;
+        private IIdentityServerConfiguration _identityServerConfiguration;
+
+        public HttpPeer(string hostAndPort, HttpClient httpClient, IWebHostBuilder builder, IOcelotConfiguration config, IIdentityServerConfiguration identityServerConfiguration)
+        {
+            _identityServerConfiguration = identityServerConfiguration;
+            _config = config;
+            Id  = hostAndPort;
+            _hostAndPort = hostAndPort;
+            _httpClient = httpClient;
+            _jsonSerializerSettings = new JsonSerializerSettings() { 
+                TypeNameHandling = TypeNameHandling.All
+            };
+            _baseSchemeUrlAndPort = builder.GetSetting(WebHostDefaults.ServerUrlsKey);
+        }
+
+        public string Id {get; private set;}
+
+        public RequestVoteResponse Request(RequestVote requestVote)
+        {
+            if(_token == null)
+            {
+                SetToken();
+            }
+
+            var json = JsonConvert.SerializeObject(requestVote, _jsonSerializerSettings);
+            var content = new StringContent(json);
+            content.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue("application/json");
+            var response = _httpClient.PostAsync($"{_hostAndPort}/administration/raft/requestvote", content).GetAwaiter().GetResult();
+            if(response.IsSuccessStatusCode)
+            {
+                return JsonConvert.DeserializeObject<RequestVoteResponse>(response.Content.ReadAsStringAsync().GetAwaiter().GetResult(), _jsonSerializerSettings);
+            }
+            else
+            {
+                return new RequestVoteResponse(false, requestVote.Term);
+            }
+        }
+
+        public AppendEntriesResponse Request(AppendEntries appendEntries)
+        {
+            try
+            {
+                if(_token == null)
+                {
+                    SetToken();
+                }                
+                var json = JsonConvert.SerializeObject(appendEntries, _jsonSerializerSettings);
+                var content = new StringContent(json);
+                content.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue("application/json");
+                var response = _httpClient.PostAsync($"{_hostAndPort}/administration/raft/appendEntries", content).GetAwaiter().GetResult();
+                if(response.IsSuccessStatusCode)
+                {
+                    return JsonConvert.DeserializeObject<AppendEntriesResponse>(response.Content.ReadAsStringAsync().GetAwaiter().GetResult(),_jsonSerializerSettings);
+                }
+                else
+                {
+                    return new AppendEntriesResponse(appendEntries.Term, false);
+                }
+            }
+            catch(Exception ex)
+            {
+                Console.WriteLine(ex);
+                return new AppendEntriesResponse(appendEntries.Term, false);
+            }
+        }
+
+        public Response<T> Request<T>(T command) where T : ICommand
+        {
+            if(_token == null)
+            {
+                SetToken();
+            }   
+            var json = JsonConvert.SerializeObject(command, _jsonSerializerSettings);
+            var content = new StringContent(json);
+            content.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue("application/json");
+            var response = _httpClient.PostAsync($"{_hostAndPort}/administration/raft/command", content).GetAwaiter().GetResult();
+            if(response.IsSuccessStatusCode)
+            {
+                return JsonConvert.DeserializeObject<OkResponse<T>>(response.Content.ReadAsStringAsync().GetAwaiter().GetResult(), _jsonSerializerSettings);
+            }
+            else 
+            {
+                return new ErrorResponse<T>(response.Content.ReadAsStringAsync().GetAwaiter().GetResult(), command);
+            }
+        }
+
+        private void SetToken()
+        {
+            var tokenUrl = $"{_baseSchemeUrlAndPort}{_config.AdministrationPath}/connect/token";
+            var formData = new List<KeyValuePair<string, string>>
+            {
+                new KeyValuePair<string, string>("client_id", _identityServerConfiguration.ApiName),
+                new KeyValuePair<string, string>("client_secret", _identityServerConfiguration.ApiSecret),
+                new KeyValuePair<string, string>("scope", _identityServerConfiguration.ApiName),
+                new KeyValuePair<string, string>("grant_type", "client_credentials")
+            };
+            var content = new FormUrlEncodedContent(formData);
+            var response = _httpClient.PostAsync(tokenUrl, content).GetAwaiter().GetResult();
+            var responseContent = response.Content.ReadAsStringAsync().GetAwaiter().GetResult();
+            response.EnsureSuccessStatusCode();
+            _token = JsonConvert.DeserializeObject<BearerToken>(responseContent);
+            _httpClient.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue(_token.TokenType, _token.AccessToken);
+        }
+    }
+}

src/Ocelot/Raft/OcelotFiniteStateMachine.cs

@@ -0,0 +1,25 @@
+using Ocelot.Configuration.Setter;
+using Rafty.FiniteStateMachine;
+using Rafty.Log;
+
+namespace Ocelot.Raft
+{
+    [ExcludeFromCoverage]
+    public class OcelotFiniteStateMachine : IFiniteStateMachine
+    {
+        private IFileConfigurationSetter _setter;
+
+        public OcelotFiniteStateMachine(IFileConfigurationSetter setter)
+        {
+            _setter = setter;
+        }
+
+        public void Handle(LogEntry log)
+        {
+            //todo - handle an error
+            //hack it to just cast as at the moment we know this is the only command :P
+            var hack = (UpdateFileConfiguration)log.CommandData;
+            _setter.Set(hack.Configuration).GetAwaiter().GetResult();;
+        }
+    }
+}

src/Ocelot/Raft/RaftController.cs

@@ -0,0 +1,84 @@
+using System;
+using System.IO;
+using System.Threading.Tasks;
+using Microsoft.AspNetCore.Authorization;
+using Microsoft.AspNetCore.Hosting;
+using Microsoft.AspNetCore.Mvc;
+using Microsoft.Extensions.Logging;
+using Newtonsoft.Json;
+using Ocelot.Logging;
+using Ocelot.Raft;
+using Rafty.Concensus;
+using Rafty.FiniteStateMachine;
+
+namespace Ocelot.Raft
+{
+    [ExcludeFromCoverage]
+    [Authorize]
+    [Route("raft")]
+    public class RaftController : Controller
+    {
+        private readonly INode _node;
+        private IOcelotLogger _logger;
+        private string _baseSchemeUrlAndPort;
+        private JsonSerializerSettings _jsonSerialiserSettings;
+
+        public RaftController(INode node, IOcelotLoggerFactory loggerFactory, IWebHostBuilder builder)
+        {
+            _jsonSerialiserSettings = new JsonSerializerSettings {
+                TypeNameHandling = TypeNameHandling.All
+            };
+            _baseSchemeUrlAndPort = builder.GetSetting(WebHostDefaults.ServerUrlsKey);
+            _logger = loggerFactory.CreateLogger<RaftController>();
+            _node = node;
+        }
+
+        [Route("appendentries")]
+        public async Task<IActionResult> AppendEntries()
+        {
+            using(var reader = new StreamReader(HttpContext.Request.Body))
+            {
+                var json = await reader.ReadToEndAsync();
+                var appendEntries = JsonConvert.DeserializeObject<AppendEntries>(json, _jsonSerialiserSettings);
+                _logger.LogDebug($"{_baseSchemeUrlAndPort}/appendentries called, my state is {_node.State.GetType().FullName}");
+                var appendEntriesResponse = _node.Handle(appendEntries);
+                return new OkObjectResult(appendEntriesResponse);
+            }
+        }
+
+        [Route("requestvote")]
+        public async Task<IActionResult> RequestVote()
+        { 
+            using(var reader = new StreamReader(HttpContext.Request.Body))
+            {
+                var json = await reader.ReadToEndAsync();
+                var requestVote = JsonConvert.DeserializeObject<RequestVote>(json, _jsonSerialiserSettings);
+                _logger.LogDebug($"{_baseSchemeUrlAndPort}/requestvote called, my state is {_node.State.GetType().FullName}");
+                var requestVoteResponse = _node.Handle(requestVote);
+                return new OkObjectResult(requestVoteResponse);
+            }
+        }
+
+        [Route("command")]
+        public async Task<IActionResult> Command()
+        { 
+            try
+            {
+                using(var reader = new StreamReader(HttpContext.Request.Body))
+                {
+                    var json = await reader.ReadToEndAsync();
+                    var command = JsonConvert.DeserializeObject<ICommand>(json, _jsonSerialiserSettings);
+                    _logger.LogDebug($"{_baseSchemeUrlAndPort}/command called, my state is {_node.State.GetType().FullName}");
+                    var commandResponse = _node.Accept(command);
+                    json = JsonConvert.SerializeObject(commandResponse, _jsonSerialiserSettings);
+                    return StatusCode(200, json);
+                }
+            }
+            catch(Exception e)
+            {
+                _logger.LogError($"THERE WAS A PROBLEM ON NODE {_node.State.CurrentState.Id}", e);
+                throw e;
+            }
+        }
+    }
+}

src/Ocelot/Raft/SqlLiteLog.cs

@@ -0,0 +1,279 @@
+using System.IO;
+using Rafty.Log;
+using Microsoft.Data.Sqlite;
+using Newtonsoft.Json;
+using System;
+using Rafty.Infrastructure;
+using System.Collections.Generic;
+
+namespace Ocelot.Raft
+{
+    [ExcludeFromCoverage]
+    public class SqlLiteLog : ILog
+    {
+        private string _path;
+        private readonly object _lock = new object();
+
+        public SqlLiteLog(NodeId nodeId)
+        {
+            _path = $"{nodeId.Id.Replace("/","").Replace(":","")}.db";
+            if(!File.Exists(_path))
+            {
+                lock(_lock)
+                {
+                    FileStream fs = File.Create(_path);
+                    fs.Dispose();
+                }
+                using(var connection = new SqliteConnection($"Data Source={_path};"))
+                {
+                    connection.Open();
+                    var sql = @"create table logs (
+                        id integer primary key,
+                        data text not null
+                    )";
+                    using(var command = new SqliteCommand(sql, connection))
+                    {
+                        var result = command.ExecuteNonQuery();
+                    }
+                }
+            }
+        }
+
+        public int LastLogIndex
+        {
+            get
+            {
+                lock(_lock)
+                {
+                    var result = 1;
+                    using(var connection = new SqliteConnection($"Data Source={_path};"))
+                    {
+                        connection.Open();
+                        var sql = @"select id from logs order by id desc limit 1";
+                        using(var command = new SqliteCommand(sql, connection))
+                        {
+                            var index = Convert.ToInt32(command.ExecuteScalar());
+                            if(index > result)
+                            {
+                                result = index;
+                            }
+                        }
+                    }
+                    return result;
+                }
+            }
+        }
+
+        public long LastLogTerm 
+        {
+            get
+            {
+                lock(_lock)
+                {
+                    long result = 0;
+                    using(var connection = new SqliteConnection($"Data Source={_path};"))
+                    {
+                        connection.Open();
+                        var sql = @"select data from logs order by id desc limit 1";
+                        using(var command = new SqliteCommand(sql, connection))
+                        {
+                            var data = Convert.ToString(command.ExecuteScalar());
+                            var jsonSerializerSettings = new JsonSerializerSettings() { 
+                                TypeNameHandling = TypeNameHandling.All
+                            };
+                            var log = JsonConvert.DeserializeObject<LogEntry>(data, jsonSerializerSettings);
+                            if(log != null && log.Term > result)
+                            {
+                                result = log.Term;
+                            }
+                        }
+                    }
+                    return result;
+                }
+            }
+        }
+
+        public int Count 
+        {
+            get 
+            {
+                lock(_lock)
+                {
+                    var result = 0;
+                    using(var connection = new SqliteConnection($"Data Source={_path};"))
+                    {
+                        connection.Open();
+                        var sql = @"select count(id) from logs";
+                        using(var command = new SqliteCommand(sql, connection))
+                        {
+                            var index = Convert.ToInt32(command.ExecuteScalar());
+                            if(index > result)
+                            {
+                                result = index;
+                            }
+                        }
+                    }
+                    return result;
+                }
+            }
+        }
+
+        public int Apply(LogEntry log)
+        {
+            lock(_lock)
+            {
+                using(var connection = new SqliteConnection($"Data Source={_path};"))
+                {
+                    connection.Open();
+                    var jsonSerializerSettings = new JsonSerializerSettings() { 
+                        TypeNameHandling = TypeNameHandling.All
+                    };
+                    var data = JsonConvert.SerializeObject(log, jsonSerializerSettings);
+                    //todo - sql injection dont copy this..
+                    var sql = $"insert into logs (data) values ('{data}')";
+                    using(var command = new SqliteCommand(sql, connection))
+                    {
+                        var result = command.ExecuteNonQuery();
+                    }
+                    
+                    sql = "select last_insert_rowid()";
+                    using(var command = new SqliteCommand(sql, connection))
+                    {
+                        var result = command.ExecuteScalar();
+                        return Convert.ToInt32(result);
+                    }   
+                }
+            }
+        }
+
+        public void DeleteConflictsFromThisLog(int index, LogEntry logEntry)
+        {
+            lock(_lock)
+            {
+                using(var connection = new SqliteConnection($"Data Source={_path};"))
+                {
+                    connection.Open();
+                    //todo - sql injection dont copy this..
+                    var sql = $"select data from logs where id = {index};";
+                    using(var command = new SqliteCommand(sql, connection))
+                    {
+                        var data = Convert.ToString(command.ExecuteScalar());
+                        var jsonSerializerSettings = new JsonSerializerSettings() { 
+                            TypeNameHandling = TypeNameHandling.All
+                        };
+                        var log = JsonConvert.DeserializeObject<LogEntry>(data, jsonSerializerSettings);
+                        if(logEntry != null && log != null && logEntry.Term != log.Term)
+                        {
+                            //todo - sql injection dont copy this..
+                            var deleteSql = $"delete from logs where id >= {index};";
+                            using(var deleteCommand = new SqliteCommand(deleteSql, connection))
+                            {
+                            var result = deleteCommand.ExecuteNonQuery();
+                            }
+                        }
+                    }
+                }
+            }
+        }
+
+        public LogEntry Get(int index)
+        {
+            lock(_lock)
+            {
+                using(var connection = new SqliteConnection($"Data Source={_path};"))
+                {
+                    connection.Open();
+                    //todo - sql injection dont copy this..
+                    var sql = $"select data from logs where id = {index}";
+                    using(var command = new SqliteCommand(sql, connection))
+                    {
+                        var data = Convert.ToString(command.ExecuteScalar());
+                        var jsonSerializerSettings = new JsonSerializerSettings() { 
+                            TypeNameHandling = TypeNameHandling.All
+                        };
+                        var log = JsonConvert.DeserializeObject<LogEntry>(data, jsonSerializerSettings);
+                        return log;
+                    }
+                }
+            }
+        }
+
+        public System.Collections.Generic.List<(int index, LogEntry logEntry)> GetFrom(int index)
+        {
+            lock(_lock)
+            {
+                var logsToReturn = new List<(int, LogEntry)>();
+
+                using(var connection = new SqliteConnection($"Data Source={_path};"))
+                {
+                    connection.Open();
+                    //todo - sql injection dont copy this..
+                    var sql = $"select id, data from logs where id >= {index}";
+                    using(var command = new SqliteCommand(sql, connection))
+                    {
+                        using(var reader = command.ExecuteReader())
+                        {
+                            while(reader.Read())
+                            {
+                                var id = Convert.ToInt32(reader[0]);
+                                var data = (string)reader[1];
+                                var jsonSerializerSettings = new JsonSerializerSettings() { 
+                                    TypeNameHandling = TypeNameHandling.All
+                                };
+                                var log = JsonConvert.DeserializeObject<LogEntry>(data, jsonSerializerSettings);
+                                logsToReturn.Add((id, log));
+
+                            }
+                        }
+                    }
+                }
+
+                return logsToReturn;
+            }
+          
+        }
+
+        public long GetTermAtIndex(int index)
+        {
+            lock(_lock)
+            {
+                long result = 0;
+                using(var connection = new SqliteConnection($"Data Source={_path};"))
+                {
+                    connection.Open();
+                    //todo - sql injection dont copy this..
+                    var sql = $"select data from logs where id = {index}";
+                    using(var command = new SqliteCommand(sql, connection))
+                    {
+                        var data = Convert.ToString(command.ExecuteScalar());
+                        var jsonSerializerSettings = new JsonSerializerSettings() { 
+                            TypeNameHandling = TypeNameHandling.All
+                        };
+                        var log = JsonConvert.DeserializeObject<LogEntry>(data, jsonSerializerSettings);
+                        if(log != null && log.Term > result)
+                        {
+                            result = log.Term;
+                        }
+                    }
+                }
+                return result;
+            }
+        }
+        public void Remove(int indexOfCommand)
+        {
+            lock(_lock)
+            {
+                using(var connection = new SqliteConnection($"Data Source={_path};"))
+                {
+                    connection.Open();
+                    //todo - sql injection dont copy this..
+                    var deleteSql = $"delete from logs where id >= {indexOfCommand};";
+                    using(var deleteCommand = new SqliteCommand(deleteSql, connection))
+                    {
+                        var result = deleteCommand.ExecuteNonQuery();
+                    }
+                }
+            }
+        }
+    }
+}

src/Ocelot/Raft/UpdateFileConfiguration.cs

@@ -0,0 +1,15 @@
+using Ocelot.Configuration.File;
+using Rafty.FiniteStateMachine;
+
+namespace Ocelot.Raft
+{
+    public class UpdateFileConfiguration : ICommand
+    {
+        public UpdateFileConfiguration(FileConfiguration configuration)
+        {
+            Configuration = configuration;
+        }
+        
+        public FileConfiguration Configuration {get;private set;}
+    }
+}

test/Ocelot.AcceptanceTests/AcceptanceTestsStartup.cs

@@ -12,9 +12,9 @@ using Ocelot.AcceptanceTests.Caching;
 
 namespace Ocelot.AcceptanceTests
 {
-    public class Startup
+    public class AcceptanceTestsStartup
     {
-        public Startup(IHostingEnvironment env)
+        public AcceptanceTestsStartup(IHostingEnvironment env)
         {
             var builder = new ConfigurationBuilder()
                 .SetBasePath(env.ContentRootPath)
@@ -41,7 +41,7 @@ namespace Ocelot.AcceptanceTests
         }
     }
 
-    public class Startup_WithCustomCacheHandle : Startup
+    public class Startup_WithCustomCacheHandle : AcceptanceTestsStartup
     {
         public Startup_WithCustomCacheHandle(IHostingEnvironment env) : base(env) { }
 
@@ -60,7 +60,7 @@ namespace Ocelot.AcceptanceTests
         }
     }
 
-    public class Startup_WithConsul_And_CustomCacheHandle : Startup
+    public class Startup_WithConsul_And_CustomCacheHandle : AcceptanceTestsStartup
     {
         public Startup_WithConsul_And_CustomCacheHandle(IHostingEnvironment env) : base(env) { }
 

test/Ocelot.AcceptanceTests/Steps.cs

@@ -83,7 +83,7 @@ namespace Ocelot.AcceptanceTests
             });
 
             _ocelotServer = new TestServer(_webHostBuilder
-                .UseStartup<Startup>());
+                .UseStartup<AcceptanceTestsStartup>());
 
             _ocelotClient = _ocelotServer.CreateClient();
         }
@@ -103,7 +103,7 @@ namespace Ocelot.AcceptanceTests
             });
 
             _ocelotServer = new TestServer(_webHostBuilder
-                .UseStartup<Startup>());
+                .UseStartup<AcceptanceTestsStartup>());
 
             _ocelotClient = _ocelotServer.CreateClient();
         }
@@ -157,7 +157,6 @@ namespace Ocelot.AcceptanceTests
         {
             var response = JsonConvert.DeserializeObject<FileConfiguration>(_response.Content.ReadAsStringAsync().Result);
 
-            response.GlobalConfiguration.AdministrationPath.ShouldBe(expected.GlobalConfiguration.AdministrationPath);
             response.GlobalConfiguration.RequestIdKey.ShouldBe(expected.GlobalConfiguration.RequestIdKey);
             response.GlobalConfiguration.ServiceDiscoveryProvider.Host.ShouldBe(expected.GlobalConfiguration.ServiceDiscoveryProvider.Host);
             response.GlobalConfiguration.ServiceDiscoveryProvider.Port.ShouldBe(expected.GlobalConfiguration.ServiceDiscoveryProvider.Port);

test/Ocelot.IntegrationTests/AdministrationTests.cs

@@ -39,13 +39,7 @@ namespace Ocelot.IntegrationTests
         [Fact]
         public void should_return_response_401_with_call_re_routes_controller()
         {
-            var configuration = new FileConfiguration
+            var configuration = new FileConfiguration();
-            {
-                GlobalConfiguration = new FileGlobalConfiguration
-                {
-                    AdministrationPath = "/administration"
-                }
-            };
 
             this.Given(x => GivenThereIsAConfiguration(configuration))
                 .And(x => GivenOcelotIsRunning())
@@ -57,13 +51,7 @@ namespace Ocelot.IntegrationTests
          [Fact]
          public void should_return_response_200_with_call_re_routes_controller()
          {
-             var configuration = new FileConfiguration
+            var configuration = new FileConfiguration();
-             { 
-                 GlobalConfiguration = new FileGlobalConfiguration
-                 {
-                     AdministrationPath = "/administration"
-                 }
-             };
 
              this.Given(x => GivenThereIsAConfiguration(configuration))
                  .And(x => GivenOcelotIsRunning())
@@ -77,13 +65,7 @@ namespace Ocelot.IntegrationTests
         [Fact]
         public void should_be_able_to_use_token_from_ocelot_a_on_ocelot_b()
         {
-            var configuration = new FileConfiguration
+            var configuration = new FileConfiguration();
-            {
-                GlobalConfiguration = new FileGlobalConfiguration
-                {
-                    AdministrationPath = "/administration"
-                }
-            };
 
             this.Given(x => GivenThereIsAConfiguration(configuration))
                 .And(x => GivenIdentityServerSigningEnvironmentalVariablesAreSet())
@@ -102,7 +84,6 @@ namespace Ocelot.IntegrationTests
             {
                 GlobalConfiguration = new FileGlobalConfiguration
                 {
-                    AdministrationPath = "/administration",
                     RequestIdKey = "RequestId",
                     ServiceDiscoveryProvider = new FileServiceDiscoveryProvider
                     {
@@ -160,7 +141,6 @@ namespace Ocelot.IntegrationTests
             {
                 GlobalConfiguration = new FileGlobalConfiguration
                 {
-                    AdministrationPath = "/administration"
                 },
                 ReRoutes = new List<FileReRoute>()
                 {
@@ -189,7 +169,6 @@ namespace Ocelot.IntegrationTests
             {
                 GlobalConfiguration = new FileGlobalConfiguration
                 {
-                    AdministrationPath = "/administration"
                 },
                 ReRoutes = new List<FileReRoute>()
                 {
@@ -234,7 +213,6 @@ namespace Ocelot.IntegrationTests
             {
                 GlobalConfiguration = new FileGlobalConfiguration
                 {
-                    AdministrationPath = "/administration"
                 },
                 ReRoutes = new List<FileReRoute>()
                 {
@@ -289,7 +267,7 @@ namespace Ocelot.IntegrationTests
                .ConfigureServices(x => {
                    x.AddSingleton(_webHostBuilderTwo);
                })
-               .UseStartup<Startup>();
+               .UseStartup<IntegrationTestsStartup>();
 
             _builderTwo = _webHostBuilderTwo.Build();
 
@@ -327,7 +305,6 @@ namespace Ocelot.IntegrationTests
         {
             var response = JsonConvert.DeserializeObject<FileConfiguration>(_response.Content.ReadAsStringAsync().Result);
 
-            response.GlobalConfiguration.AdministrationPath.ShouldBe(expected.GlobalConfiguration.AdministrationPath);
             response.GlobalConfiguration.RequestIdKey.ShouldBe(expected.GlobalConfiguration.RequestIdKey);
             response.GlobalConfiguration.ServiceDiscoveryProvider.Host.ShouldBe(expected.GlobalConfiguration.ServiceDiscoveryProvider.Host);
             response.GlobalConfiguration.ServiceDiscoveryProvider.Port.ShouldBe(expected.GlobalConfiguration.ServiceDiscoveryProvider.Port);
@@ -356,9 +333,7 @@ namespace Ocelot.IntegrationTests
                 new KeyValuePair<string, string>("client_id", "admin"),
                 new KeyValuePair<string, string>("client_secret", "secret"),
                 new KeyValuePair<string, string>("scope", "admin"),
-                new KeyValuePair<string, string>("username", "admin"),
+                new KeyValuePair<string, string>("grant_type", "client_credentials")
-                new KeyValuePair<string, string>("password", "secret"),
-                new KeyValuePair<string, string>("grant_type", "password")
             };
             var content = new FormUrlEncodedContent(formData);
 
@@ -380,7 +355,7 @@ namespace Ocelot.IntegrationTests
                 .ConfigureServices(x => {
                     x.AddSingleton(_webHostBuilder);
                 })
-                .UseStartup<Startup>();
+                .UseStartup<IntegrationTestsStartup>();
 
               _builder = _webHostBuilder.Build();
 

test/Ocelot.IntegrationTests/IntegrationTestsStartup.cs

@@ -11,9 +11,9 @@ using ConfigurationBuilder = Microsoft.Extensions.Configuration.ConfigurationBui
 
 namespace Ocelot.IntegrationTests
 {
-    public class Startup
+    public class IntegrationTestsStartup
     {
-        public Startup(IHostingEnvironment env)
+        public IntegrationTestsStartup(IHostingEnvironment env)
         {
             var builder = new ConfigurationBuilder()
                 .SetBasePath(env.ContentRootPath)
@@ -38,7 +38,9 @@ namespace Ocelot.IntegrationTests
                     .WithDictionaryHandle();
             };
 
-            services.AddOcelot(Configuration);
+            services.AddOcelot(Configuration)
+                .AddCacheManager(settings)
+                .AddAdministration("/administration", "secret");
         }
 
         public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)

test/Ocelot.IntegrationTests/Ocelot.IntegrationTests.csproj

@@ -1,5 +1,4 @@
-<Project Sdk="Microsoft.NET.Sdk">
+<Project Sdk="Microsoft.NET.Sdk">
-
   <PropertyGroup>
     <VersionPrefix>0.0.0-dev</VersionPrefix>
     <TargetFramework>netcoreapp2.0</TargetFramework>
@@ -13,39 +12,35 @@
     <GenerateAssemblyCompanyAttribute>false</GenerateAssemblyCompanyAttribute>
     <GenerateAssemblyProductAttribute>false</GenerateAssemblyProductAttribute>
   </PropertyGroup>
-
   <ItemGroup>
-    <None Update="configuration.json;appsettings.json;idsrv3test.pfx">
+    <None Update="peers.json;configuration.json;appsettings.json;idsrv3test.pfx">
       <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
     </None>
   </ItemGroup>
-
   <ItemGroup>
-  <Service Include="{82a7f48d-3b50-4b1e-b82e-3ada8210c358}" />
+    <Service Include="{82a7f48d-3b50-4b1e-b82e-3ada8210c358}"/>
   </ItemGroup>
-
   <ItemGroup>
-    <ProjectReference Include="..\..\src\Ocelot\Ocelot.csproj" />
+    <ProjectReference Include="..\..\src\Ocelot\Ocelot.csproj"/>
-    <ProjectReference Include="..\Ocelot.ManualTest\Ocelot.ManualTest.csproj" />
   </ItemGroup>
-
   <ItemGroup>
-    <PackageReference Include="Microsoft.AspNetCore.All" Version="2.0.0" />
+    <PackageReference Include="Microsoft.AspNetCore.All" Version="2.0.0"/>
-    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.5.0-preview-20171031-01" />
+    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.5.0-preview-20171031-01"/>
-    <PackageReference Include="xunit.runner.visualstudio" Version="2.3.1" />
+    <PackageReference Include="xunit.runner.visualstudio" Version="2.3.1"/>
-    <PackageReference Include="Microsoft.Extensions.Configuration.EnvironmentVariables" Version="2.0.0" />
+    <PackageReference Include="Microsoft.Extensions.Configuration.EnvironmentVariables" Version="2.0.0"/>
-    <PackageReference Include="Microsoft.Extensions.Configuration.FileExtensions" Version="2.0.0" />
+    <PackageReference Include="Microsoft.Extensions.Configuration.FileExtensions" Version="2.0.0"/>
-    <PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="2.0.0" />
+    <PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="2.0.0"/>
-    <PackageReference Include="Microsoft.Extensions.Logging" Version="2.0.0" />
+    <PackageReference Include="Microsoft.Extensions.Logging" Version="2.0.0"/>
-    <PackageReference Include="Microsoft.Extensions.Logging.Console" Version="2.0.0" />
+    <PackageReference Include="Microsoft.Extensions.Logging.Console" Version="2.0.0"/>
-    <PackageReference Include="Microsoft.Extensions.Logging.Debug" Version="2.0.0" />
+    <PackageReference Include="Microsoft.Extensions.Logging.Debug" Version="2.0.0"/>
-    <PackageReference Include="Microsoft.Extensions.Options.ConfigurationExtensions" Version="2.0.0" />
+    <PackageReference Include="Microsoft.Extensions.Options.ConfigurationExtensions" Version="2.0.0"/>
-    <PackageReference Include="Microsoft.DotNet.InternalAbstractions" Version="1.0.500-preview2-1-003177" />
+    <PackageReference Include="Microsoft.DotNet.InternalAbstractions" Version="1.0.500-preview2-1-003177"/>
-    <PackageReference Include="xunit" Version="2.3.1" />
+    <PackageReference Include="xunit" Version="2.3.1"/>
-    <PackageReference Include="IdentityServer4" Version="2.0.2" />
+    <PackageReference Include="IdentityServer4" Version="2.0.2"/>
-    <PackageReference Include="Shouldly" Version="3.0.0-beta0003" />
+    <PackageReference Include="Shouldly" Version="3.0.0-beta0003"/>
-    <PackageReference Include="TestStack.BDDfy" Version="4.3.2" />
+    <PackageReference Include="TestStack.BDDfy" Version="4.3.2"/>
-    <PackageReference Include="Consul" Version="0.7.2.3" />
+    <PackageReference Include="Consul" Version="0.7.2.3"/>
+    <PackageReference Include="Rafty" Version="0.4.2"/>
+    <PackageReference Include="Microsoft.Data.SQLite" Version="2.0.0"/>
   </ItemGroup>
-
 </Project>

test/Ocelot.IntegrationTests/RaftStartup.cs

@@ -0,0 +1,55 @@
+using System;
+using System.IO;
+using System.Linq;
+using Microsoft.AspNetCore.Builder;
+using Microsoft.AspNetCore.Hosting;
+using Microsoft.AspNetCore.Http;
+using Microsoft.Extensions.Configuration;
+using Microsoft.Extensions.DependencyInjection;
+using Microsoft.Extensions.Logging;
+using Newtonsoft.Json;
+using Ocelot.DependencyInjection;
+using Ocelot.Middleware;
+using Ocelot.Raft;
+using Rafty.Concensus;
+using Rafty.FiniteStateMachine;
+using Rafty.Infrastructure;
+using Rafty.Log;
+using ConfigurationBuilder = Microsoft.Extensions.Configuration.ConfigurationBuilder;
+
+namespace Ocelot.IntegrationTests
+{
+    public class RaftStartup
+    {
+        public RaftStartup(IHostingEnvironment env)
+        {
+            var builder = new ConfigurationBuilder()
+                .SetBasePath(env.ContentRootPath)
+                .AddJsonFile("appsettings.json", optional: true, reloadOnChange: true)
+                .AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true)
+                .AddJsonFile("peers.json", optional: true, reloadOnChange: true)
+                .AddJsonFile("configuration.json")
+                .AddEnvironmentVariables();
+
+            Configuration = builder.Build();
+        }
+
+        public IConfigurationRoot Configuration { get; }
+
+        public virtual void ConfigureServices(IServiceCollection services)
+        {
+            services
+                .AddOcelot(Configuration)
+                .AddAdministration("/administration", "secret")
+                .AddRafty();
+        }
+
+        public virtual void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
+        {
+
+            //this is from Ocelot...so we need to move stuff below into it...
+            loggerFactory.AddConsole(Configuration.GetSection("Logging"));
+            app.UseOcelot().Wait();
+        }
+    }
+}

test/Ocelot.IntegrationTests/RaftTests.cs

@@ -0,0 +1,431 @@
+using System;
+using System.Collections.Generic;
+using System.Diagnostics;
+using System.IO;
+using System.Linq;
+using System.Net.Http;
+using System.Net.Http.Headers;
+using System.Text;
+using System.Threading;
+using System.Threading.Tasks;
+using Microsoft.AspNetCore.Hosting;
+using Microsoft.Extensions.DependencyInjection;
+using Newtonsoft.Json;
+using Ocelot.Configuration.File;
+using Ocelot.Raft;
+using Rafty.Concensus;
+using Rafty.FiniteStateMachine;
+using Rafty.Infrastructure;
+using Shouldly;
+using Xunit;
+using static Rafty.Infrastructure.Wait;
+using Microsoft.Data.Sqlite;
+
+namespace Ocelot.IntegrationTests
+{
+    public class RaftTests : IDisposable
+    {
+        private List<IWebHost> _builders;
+        private List<IWebHostBuilder> _webHostBuilders;
+        private List<Thread> _threads;
+        private FilePeers _peers;
+        private HttpClient _httpClient;
+        private HttpClient _httpClientForAssertions;
+        private string _ocelotBaseUrl;
+        private BearerToken _token;
+        private HttpResponseMessage _response;
+        private static object _lock = new object();
+
+        public RaftTests()
+        {
+            _httpClientForAssertions = new HttpClient();
+            _httpClient = new HttpClient();
+            _ocelotBaseUrl = "http://localhost:5000";
+            _httpClient.BaseAddress = new Uri(_ocelotBaseUrl);
+            _webHostBuilders = new List<IWebHostBuilder>();
+            _builders = new List<IWebHost>();
+            _threads = new List<Thread>();
+        }
+        public void Dispose()
+        {
+            foreach (var builder in _builders)
+            {
+                builder?.Dispose();
+            }
+
+            foreach (var peer in _peers.Peers)
+            {
+                File.Delete(peer.HostAndPort.Replace("/","").Replace(":",""));
+                File.Delete($"{peer.HostAndPort.Replace("/","").Replace(":","")}.db");
+            }
+        }
+
+        [Fact]
+        public void should_persist_command_to_five_servers()
+        {
+             var configuration = new FileConfiguration
+             { 
+                 GlobalConfiguration = new FileGlobalConfiguration
+                 {
+                 }
+             };
+
+            var updatedConfiguration = new FileConfiguration
+            {
+                GlobalConfiguration = new FileGlobalConfiguration
+                {
+                },
+                ReRoutes = new List<FileReRoute>()
+                {
+                    new FileReRoute()
+                    {
+                        DownstreamHost = "127.0.0.1",
+                        DownstreamPort = 80,
+                        DownstreamScheme = "http",
+                        DownstreamPathTemplate = "/geoffrey",
+                        UpstreamHttpMethod = new List<string> { "get" },
+                        UpstreamPathTemplate = "/"
+                    },
+                    new FileReRoute()
+                    {
+                        DownstreamHost = "123.123.123",
+                        DownstreamPort = 443,
+                        DownstreamScheme = "https",
+                        DownstreamPathTemplate = "/blooper/{productId}",
+                        UpstreamHttpMethod = new List<string> { "post" },
+                        UpstreamPathTemplate = "/test"
+                    }
+                }
+            };
+             
+            var command = new UpdateFileConfiguration(updatedConfiguration);
+            GivenThereIsAConfiguration(configuration);
+            GivenFiveServersAreRunning();
+            GivenALeaderIsElected();
+            GivenIHaveAnOcelotToken("/administration");
+            WhenISendACommandIntoTheCluster(command);
+            ThenTheCommandIsReplicatedToAllStateMachines(command);
+        }
+
+        [Fact]
+        public void should_persist_command_to_five_servers_when_using_administration_api()
+        {
+             var configuration = new FileConfiguration
+             { 
+             };
+
+            var updatedConfiguration = new FileConfiguration
+            {
+                ReRoutes = new List<FileReRoute>()
+                {
+                    new FileReRoute()
+                    {
+                        DownstreamHost = "127.0.0.1",
+                        DownstreamPort = 80,
+                        DownstreamScheme = "http",
+                        DownstreamPathTemplate = "/geoffrey",
+                        UpstreamHttpMethod = new List<string> { "get" },
+                        UpstreamPathTemplate = "/"
+                    },
+                    new FileReRoute()
+                    {
+                        DownstreamHost = "123.123.123",
+                        DownstreamPort = 443,
+                        DownstreamScheme = "https",
+                        DownstreamPathTemplate = "/blooper/{productId}",
+                        UpstreamHttpMethod = new List<string> { "post" },
+                        UpstreamPathTemplate = "/test"
+                    }
+                }
+            };
+             
+            var command = new UpdateFileConfiguration(updatedConfiguration);
+            GivenThereIsAConfiguration(configuration);
+            GivenFiveServersAreRunning();
+            GivenALeaderIsElected();
+            GivenIHaveAnOcelotToken("/administration");
+            GivenIHaveAddedATokenToMyRequest();
+            WhenIPostOnTheApiGateway("/administration/configuration", updatedConfiguration);
+            ThenTheCommandIsReplicatedToAllStateMachines(command);
+        }
+
+        private void WhenISendACommandIntoTheCluster(UpdateFileConfiguration command)
+        {
+            var p = _peers.Peers.First();
+            var json = JsonConvert.SerializeObject(command,new JsonSerializerSettings() { 
+                TypeNameHandling = TypeNameHandling.All
+            });
+            var httpContent = new StringContent(json);
+            httpContent.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue("application/json");
+            using(var httpClient = new HttpClient())
+            {
+                httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", _token.AccessToken);
+                var response = httpClient.PostAsync($"{p.HostAndPort}/administration/raft/command", httpContent).GetAwaiter().GetResult();
+                response.EnsureSuccessStatusCode();
+                var content = response.Content.ReadAsStringAsync().GetAwaiter().GetResult();
+                var result = JsonConvert.DeserializeObject<OkResponse<UpdateFileConfiguration>>(content);
+                result.Command.Configuration.ReRoutes.Count.ShouldBe(2);
+            }
+
+            //dirty sleep to make sure command replicated...
+            var stopwatch = Stopwatch.StartNew();
+            while(stopwatch.ElapsedMilliseconds < 10000)
+            {
+
+            }
+        }
+
+        private void ThenTheCommandIsReplicatedToAllStateMachines(UpdateFileConfiguration expected)
+        {
+            //dirty sleep to give a chance to replicate...
+            var stopwatch = Stopwatch.StartNew();
+            while(stopwatch.ElapsedMilliseconds < 2000)
+            {
+
+            }
+            
+             bool CommandCalledOnAllStateMachines()
+            {
+                try
+                {
+                    var passed = 0;
+                    foreach (var peer in _peers.Peers)
+                    {
+                        var path = $"{peer.HostAndPort.Replace("/","").Replace(":","")}.db";
+                        using(var connection = new SqliteConnection($"Data Source={path};"))
+                        {
+                            connection.Open();
+                            var sql = @"select count(id) from logs";
+                            using(var command = new SqliteCommand(sql, connection))
+                            {
+                                var index = Convert.ToInt32(command.ExecuteScalar());
+                                index.ShouldBe(1);
+                            }
+                        }
+                        _httpClientForAssertions.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", _token.AccessToken);
+                        var result = _httpClientForAssertions.GetAsync($"{peer.HostAndPort}/administration/configuration").Result;
+                        var json = result.Content.ReadAsStringAsync().Result;
+                        var response = JsonConvert.DeserializeObject<FileConfiguration>(json, new JsonSerializerSettings{TypeNameHandling = TypeNameHandling.All});
+                        response.GlobalConfiguration.RequestIdKey.ShouldBe(expected.Configuration.GlobalConfiguration.RequestIdKey);
+                        response.GlobalConfiguration.ServiceDiscoveryProvider.Host.ShouldBe(expected.Configuration.GlobalConfiguration.ServiceDiscoveryProvider.Host);
+                        response.GlobalConfiguration.ServiceDiscoveryProvider.Port.ShouldBe(expected.Configuration.GlobalConfiguration.ServiceDiscoveryProvider.Port);
+
+                        for (var i = 0; i < response.ReRoutes.Count; i++)
+                        {
+                            response.ReRoutes[i].DownstreamHost.ShouldBe(expected.Configuration.ReRoutes[i].DownstreamHost);
+                            response.ReRoutes[i].DownstreamPathTemplate.ShouldBe(expected.Configuration.ReRoutes[i].DownstreamPathTemplate);
+                            response.ReRoutes[i].DownstreamPort.ShouldBe(expected.Configuration.ReRoutes[i].DownstreamPort);
+                            response.ReRoutes[i].DownstreamScheme.ShouldBe(expected.Configuration.ReRoutes[i].DownstreamScheme);
+                            response.ReRoutes[i].UpstreamPathTemplate.ShouldBe(expected.Configuration.ReRoutes[i].UpstreamPathTemplate);
+                            response.ReRoutes[i].UpstreamHttpMethod.ShouldBe(expected.Configuration.ReRoutes[i].UpstreamHttpMethod);
+                        }
+                        passed++;
+                    }
+
+                    return passed == 5;
+                }
+                catch(Exception e)
+                {
+                    Console.WriteLine(e);
+                    return false;
+                }
+            }
+
+            var commandOnAllStateMachines = WaitFor(20000).Until(() => CommandCalledOnAllStateMachines());
+            commandOnAllStateMachines.ShouldBeTrue();   
+        }
+
+        private void ThenTheResponseShouldBe(FileConfiguration expected)
+        {
+            var response = JsonConvert.DeserializeObject<FileConfiguration>(_response.Content.ReadAsStringAsync().Result);
+            
+            response.GlobalConfiguration.RequestIdKey.ShouldBe(expected.GlobalConfiguration.RequestIdKey);
+            response.GlobalConfiguration.ServiceDiscoveryProvider.Host.ShouldBe(expected.GlobalConfiguration.ServiceDiscoveryProvider.Host);
+            response.GlobalConfiguration.ServiceDiscoveryProvider.Port.ShouldBe(expected.GlobalConfiguration.ServiceDiscoveryProvider.Port);
+
+            for (var i = 0; i < response.ReRoutes.Count; i++)
+            {
+                response.ReRoutes[i].DownstreamHost.ShouldBe(expected.ReRoutes[i].DownstreamHost);
+                response.ReRoutes[i].DownstreamPathTemplate.ShouldBe(expected.ReRoutes[i].DownstreamPathTemplate);
+                response.ReRoutes[i].DownstreamPort.ShouldBe(expected.ReRoutes[i].DownstreamPort);
+                response.ReRoutes[i].DownstreamScheme.ShouldBe(expected.ReRoutes[i].DownstreamScheme);
+                response.ReRoutes[i].UpstreamPathTemplate.ShouldBe(expected.ReRoutes[i].UpstreamPathTemplate);
+                response.ReRoutes[i].UpstreamHttpMethod.ShouldBe(expected.ReRoutes[i].UpstreamHttpMethod);
+            }
+        }
+
+        private void WhenIGetUrlOnTheApiGateway(string url)
+        {
+            _response = _httpClient.GetAsync(url).Result;
+        }
+
+        private void WhenIPostOnTheApiGateway(string url, FileConfiguration updatedConfiguration)
+        {
+            var json = JsonConvert.SerializeObject(updatedConfiguration);
+            var content = new StringContent(json);
+            content.Headers.ContentType = new MediaTypeHeaderValue("application/json");
+            _response = _httpClient.PostAsync(url, content).Result;
+        }
+
+        private void GivenIHaveAddedATokenToMyRequest()
+        {
+            _httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", _token.AccessToken);
+        }
+
+        private void GivenIHaveAnOcelotToken(string adminPath)
+        {
+            var tokenUrl = $"{adminPath}/connect/token";
+            var formData = new List<KeyValuePair<string, string>>
+            {
+                new KeyValuePair<string, string>("client_id", "admin"),
+                new KeyValuePair<string, string>("client_secret", "secret"),
+                new KeyValuePair<string, string>("scope", "admin"),
+                new KeyValuePair<string, string>("grant_type", "client_credentials")
+            };
+            var content = new FormUrlEncodedContent(formData);
+
+            var response = _httpClient.PostAsync(tokenUrl, content).Result;
+            var responseContent = response.Content.ReadAsStringAsync().Result;
+            response.EnsureSuccessStatusCode();
+            _token = JsonConvert.DeserializeObject<BearerToken>(responseContent);
+            var configPath = $"{adminPath}/.well-known/openid-configuration";
+            response = _httpClient.GetAsync(configPath).Result;
+            response.EnsureSuccessStatusCode();
+        }
+
+        private void GivenThereIsAConfiguration(FileConfiguration fileConfiguration)
+        {
+            var configurationPath = $"{Directory.GetCurrentDirectory()}/configuration.json";
+
+            var jsonConfiguration = JsonConvert.SerializeObject(fileConfiguration);
+
+            if (File.Exists(configurationPath))
+            {
+                File.Delete(configurationPath);
+            }
+
+            File.WriteAllText(configurationPath, jsonConfiguration);
+
+            var text = File.ReadAllText(configurationPath);
+
+            configurationPath = $"{AppContext.BaseDirectory}/configuration.json";
+
+            if (File.Exists(configurationPath))
+            {
+                File.Delete(configurationPath);
+            }
+
+            File.WriteAllText(configurationPath, jsonConfiguration);
+
+            text = File.ReadAllText(configurationPath);
+        }
+
+        private void GivenAServerIsRunning(string url)
+        {
+            lock(_lock)
+            {
+                IWebHostBuilder webHostBuilder = new WebHostBuilder();
+                webHostBuilder.UseUrls(url)
+                    .UseKestrel()
+                    .UseContentRoot(Directory.GetCurrentDirectory())
+                    .ConfigureServices(x =>
+                    {
+                        x.AddSingleton(webHostBuilder);
+                        x.AddSingleton(new NodeId(url));
+                    })
+                    .UseStartup<RaftStartup>();
+
+                var builder = webHostBuilder.Build();
+                builder.Start();
+
+                _webHostBuilders.Add(webHostBuilder);
+                _builders.Add(builder);
+            }
+        }
+
+        private void GivenFiveServersAreRunning()
+        {
+            var bytes = File.ReadAllText("peers.json");
+            _peers = JsonConvert.DeserializeObject<FilePeers>(bytes);
+
+            foreach (var peer in _peers.Peers)
+            {
+                var thread = new Thread(() => GivenAServerIsRunning(peer.HostAndPort));
+                thread.Start();
+                _threads.Add(thread);
+            }
+        }
+
+        private void GivenALeaderIsElected()
+        {
+            //dirty sleep to make sure we have a leader
+            var stopwatch = Stopwatch.StartNew();
+            while(stopwatch.ElapsedMilliseconds < 20000)
+            {
+
+            }
+        }
+
+        private void WhenISendACommandIntoTheCluster(FakeCommand command)
+        {
+            var p = _peers.Peers.First();
+            var json = JsonConvert.SerializeObject(command,new JsonSerializerSettings() { 
+                TypeNameHandling = TypeNameHandling.All
+            });
+            var httpContent = new StringContent(json);
+            httpContent.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue("application/json");
+            using(var httpClient = new HttpClient())
+            {
+                httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", _token.AccessToken);
+                var response = httpClient.PostAsync($"{p.HostAndPort}/administration/raft/command", httpContent).GetAwaiter().GetResult();
+                response.EnsureSuccessStatusCode();
+                var content = response.Content.ReadAsStringAsync().GetAwaiter().GetResult();
+                var result = JsonConvert.DeserializeObject<OkResponse<FakeCommand>>(content);
+                result.Command.Value.ShouldBe(command.Value);
+            }
+
+            //dirty sleep to make sure command replicated...
+            var stopwatch = Stopwatch.StartNew();
+            while(stopwatch.ElapsedMilliseconds < 10000)
+            {
+
+            }
+        }
+
+        private void ThenTheCommandIsReplicatedToAllStateMachines(FakeCommand command)
+        {
+            //dirty sleep to give a chance to replicate...
+            var stopwatch = Stopwatch.StartNew();
+            while(stopwatch.ElapsedMilliseconds < 2000)
+            {
+
+            }
+            
+             bool CommandCalledOnAllStateMachines()
+            {
+                try
+                {
+                    var passed = 0;
+                    foreach (var peer in _peers.Peers)
+                    {
+                        string fsmData;
+                        fsmData = File.ReadAllText(peer.HostAndPort.Replace("/","").Replace(":",""));
+                        fsmData.ShouldNotBeNullOrEmpty();
+                        var fakeCommand = JsonConvert.DeserializeObject<FakeCommand>(fsmData);
+                        fakeCommand.Value.ShouldBe(command.Value);
+                        passed++;
+                    }
+
+                    return passed == 5;
+                }
+                catch(Exception e)
+                {
+                    return false;
+                }
+            }
+
+            var commandOnAllStateMachines = WaitFor(20000).Until(() => CommandCalledOnAllStateMachines());
+            commandOnAllStateMachines.ShouldBeTrue();   
+        }
+    }
+}

test/Ocelot.IntegrationTests/ThreadSafeHeadersTests.cs

@@ -95,7 +95,7 @@ namespace Ocelot.IntegrationTests
                 {
                     x.AddSingleton(_webHostBuilder);
                 })
-                .UseStartup<Startup>();
+                .UseStartup<IntegrationTestsStartup>();
 
             _builder = _webHostBuilder.Build();
 

test/Ocelot.IntegrationTests/peers.json

@@ -0,0 +1,18 @@
+{
+	"Peers": [{
+			"HostAndPort": "http://localhost:5000"
+		},
+		{
+			"HostAndPort": "http://localhost:5002"
+		},
+		{
+			"HostAndPort": "http://localhost:5003"
+		},
+		{
+			"HostAndPort": "http://localhost:5004"
+		},
+		{
+			"HostAndPort": "http://localhost:5001"
+		}
+	]
+}

test/Ocelot.ManualTest/ManualTestStartup.cs

@@ -11,9 +11,9 @@ using ConfigurationBuilder = Microsoft.Extensions.Configuration.ConfigurationBui
 
 namespace Ocelot.ManualTest
 {
-    public class Startup
+    public class ManualTestStartup
     {
-        public Startup(IHostingEnvironment env)
+        public ManualTestStartup(IHostingEnvironment env)
         {
             var builder = new ConfigurationBuilder()
                 .SetBasePath(env.ContentRootPath)
@@ -45,7 +45,8 @@ namespace Ocelot.ManualTest
                     x.Audience = "test";
                 });
 
-            services.AddOcelot(Configuration);
+            services.AddOcelot(Configuration)
+                    .AddAdministration("/administration", "secret");
         }
 
         public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)

test/Ocelot.ManualTest/Program.cs

@@ -15,7 +15,7 @@ namespace Ocelot.ManualTest
             builder.UseKestrel()
                 .UseContentRoot(Directory.GetCurrentDirectory())
                 .UseIISIntegration()
-                .UseStartup<Startup>();                
+                .UseStartup<ManualTestStartup>();                
             var host = builder.Build();
             host.Run();
         }

test/Ocelot.ManualTest/configuration.json

@@ -300,12 +300,11 @@
       "DownstreamHost": "www.bbc.co.uk",
       "DownstreamPort": 80,
       "UpstreamPathTemplate": "/bbc/",
-      "UpstreamHttpMethod": [ "Get" ],
+      "UpstreamHttpMethod": [ "Get" ]
     }
   ],
 
   "GlobalConfiguration": {
-    "RequestIdKey": "OcRequestId",
+    "RequestIdKey": "OcRequestId"
-    "AdministrationPath": "/administration"
   }
 }

test/Ocelot.UnitTests/Configuration/FileConfigurationCreatorTests.cs

@@ -15,6 +15,7 @@ using Xunit;
 
 namespace Ocelot.UnitTests.Configuration
 {
+    using Ocelot.DependencyInjection;
     using Ocelot.Errors;
     using Ocelot.UnitTests.TestData;
 
@@ -36,6 +37,7 @@ namespace Ocelot.UnitTests.Configuration
         private Mock<IRateLimitOptionsCreator> _rateLimitOptions;
         private Mock<IRegionCreator> _regionCreator;
         private Mock<IHttpHandlerOptionsCreator> _httpHandlerOptionsCreator;
+        private Mock<IAdministrationPath> _adminPath;
 
         public FileConfigurationCreatorTests()
         {
@@ -52,13 +54,23 @@ namespace Ocelot.UnitTests.Configuration
             _rateLimitOptions = new Mock<IRateLimitOptionsCreator>();
             _regionCreator = new Mock<IRegionCreator>();
             _httpHandlerOptionsCreator = new Mock<IHttpHandlerOptionsCreator>();
+            _adminPath = new Mock<IAdministrationPath>();
 
             _ocelotConfigurationCreator = new FileOcelotConfigurationCreator( 
-                _fileConfig.Object, _validator.Object, _logger.Object,
+                _fileConfig.Object,
+                _validator.Object, 
+                _logger.Object,
                 _claimsToThingCreator.Object,
-                _authOptionsCreator.Object, _upstreamTemplatePatternCreator.Object, _requestIdKeyCreator.Object,
+                _authOptionsCreator.Object, 
-                _serviceProviderConfigCreator.Object, _qosOptionsCreator.Object, _fileReRouteOptionsCreator.Object,
+                _upstreamTemplatePatternCreator.Object,
-                _rateLimitOptions.Object, _regionCreator.Object, _httpHandlerOptionsCreator.Object);
+                _requestIdKeyCreator.Object,
+                _serviceProviderConfigCreator.Object,
+                _qosOptionsCreator.Object,
+                _fileReRouteOptionsCreator.Object,
+                _rateLimitOptions.Object,
+                _regionCreator.Object,
+                _httpHandlerOptionsCreator.Object,
+                _adminPath.Object);
         }
 
         [Fact]

test/Ocelot.UnitTests/Configuration/FileConfigurationRepositoryTests.cs

@@ -91,7 +91,6 @@ namespace Ocelot.UnitTests.Configuration
 
         private void ThenTheConfigurationIsStoredAs(FileConfiguration expected)
         {
-            _result.GlobalConfiguration.AdministrationPath.ShouldBe(expected.GlobalConfiguration.AdministrationPath);
             _result.GlobalConfiguration.RequestIdKey.ShouldBe(expected.GlobalConfiguration.RequestIdKey);
             _result.GlobalConfiguration.ServiceDiscoveryProvider.Host.ShouldBe(expected.GlobalConfiguration.ServiceDiscoveryProvider.Host);
             _result.GlobalConfiguration.ServiceDiscoveryProvider.Port.ShouldBe(expected.GlobalConfiguration.ServiceDiscoveryProvider.Port);
@@ -126,7 +125,6 @@ namespace Ocelot.UnitTests.Configuration
 
         private void ThenTheFollowingIsReturned(FileConfiguration expected)
         {
-            _result.GlobalConfiguration.AdministrationPath.ShouldBe(expected.GlobalConfiguration.AdministrationPath);
             _result.GlobalConfiguration.RequestIdKey.ShouldBe(expected.GlobalConfiguration.RequestIdKey);
             _result.GlobalConfiguration.ServiceDiscoveryProvider.Host.ShouldBe(expected.GlobalConfiguration.ServiceDiscoveryProvider.Host);
             _result.GlobalConfiguration.ServiceDiscoveryProvider.Port.ShouldBe(expected.GlobalConfiguration.ServiceDiscoveryProvider.Port);
@@ -155,7 +153,6 @@ namespace Ocelot.UnitTests.Configuration
 
             var globalConfiguration = new FileGlobalConfiguration
             {
-                AdministrationPath = "asdas",
                 ServiceDiscoveryProvider = new FileServiceDiscoveryProvider
                 {
                     Port = 198,
@@ -185,7 +182,6 @@ namespace Ocelot.UnitTests.Configuration
 
             var globalConfiguration = new FileGlobalConfiguration
             {
-                AdministrationPath = "testy",
                 ServiceDiscoveryProvider = new FileServiceDiscoveryProvider
                 {
                     Port = 198,

test/Ocelot.UnitTests/Configuration/IdentityServerConfigurationCreatorTests.cs

@@ -9,7 +9,7 @@ namespace Ocelot.UnitTests.Configuration
         [Fact]
         public void happy_path_only_exists_for_test_coverage_even_uncle_bob_probably_wouldnt_test_this()
         {
-            var result = IdentityServerConfigurationCreator.GetIdentityServerConfiguration();
+            var result = IdentityServerConfigurationCreator.GetIdentityServerConfiguration("secret");
             result.ApiName.ShouldBe("admin");
         }
     }

test/Ocelot.UnitTests/Configuration/OcelotResourceOwnerPasswordValidatorTests.cs

@@ -1,117 +0,0 @@
-using Ocelot.Configuration.Authentication;
-using Xunit;
-using Shouldly;
-using TestStack.BDDfy;
-using Moq;
-using IdentityServer4.Validation;
-using Ocelot.Configuration.Provider;
-using System.Collections.Generic;
-
-namespace Ocelot.UnitTests.Configuration
-{
-    public class OcelotResourceOwnerPasswordValidatorTests
-    {
-        private OcelotResourceOwnerPasswordValidator _validator;
-        private Mock<IHashMatcher> _matcher;
-        private string _userName;
-        private string _password;
-        private ResourceOwnerPasswordValidationContext _context;
-        private Mock<IIdentityServerConfiguration> _config;
-        private User _user;
-
-        public OcelotResourceOwnerPasswordValidatorTests()
-        {
-            _matcher = new Mock<IHashMatcher>();
-            _config = new Mock<IIdentityServerConfiguration>();
-            _validator = new OcelotResourceOwnerPasswordValidator(_matcher.Object, _config.Object);
-        }
-
-        [Fact]
-        public void should_return_success()
-        {
-            this.Given(x => GivenTheUserName("tom"))
-                .And(x => GivenThePassword("password"))
-                .And(x => GivenTheUserIs(new User("sub", "tom", "xxx", "xxx")))
-                .And(x => GivenTheMatcherReturns(true))
-                .When(x => WhenIValidate())
-                .Then(x => ThenTheUserIsValidated())
-                .And(x => ThenTheMatcherIsCalledCorrectly())
-                .BDDfy();
-        }
-
-        [Fact]
-        public void should_return_fail_when_no_user()
-        {
-            this.Given(x => GivenTheUserName("bob"))
-                .And(x => GivenTheUserIs(new User("sub", "tom", "xxx", "xxx")))
-                .And(x => GivenTheMatcherReturns(true))
-                .When(x => WhenIValidate())
-                .Then(x => ThenTheUserIsNotValidated())
-                .BDDfy();
-        }
-
-        [Fact]
-        public void should_return_fail_when_password_doesnt_match()
-        {
-            this.Given(x => GivenTheUserName("tom"))
-                .And(x => GivenThePassword("password"))
-                .And(x => GivenTheUserIs(new User("sub", "tom", "xxx", "xxx")))
-                .And(x => GivenTheMatcherReturns(false))
-                .When(x => WhenIValidate())
-                .Then(x => ThenTheUserIsNotValidated())
-                .And(x => ThenTheMatcherIsCalledCorrectly())
-                .BDDfy();
-        }
-
-        private void ThenTheMatcherIsCalledCorrectly()
-        {
-            _matcher
-                .Verify(x => x.Match(_password, _user.Salt, _user.Hash), Times.Once);
-        }
-
-        private void GivenThePassword(string password)
-        {
-            _password = password;
-        }
-
-        private void GivenTheUserIs(User user)
-        {
-            _user = user;
-            _config
-                .Setup(x => x.Users)
-                .Returns(new List<User>{_user});
-        }
-
-        private void GivenTheMatcherReturns(bool expected)
-        {
-            _matcher
-                .Setup(x => x.Match(It.IsAny<string>(), It.IsAny<string>(), It.IsAny<string>()))
-                .Returns(expected);
-        }
-
-        private void GivenTheUserName(string userName)
-        {
-            _userName = userName;
-        }
-
-        private void WhenIValidate()
-        {
-            _context = new ResourceOwnerPasswordValidationContext
-            {
-                UserName = _userName,
-                Password = _password
-            };
-            _validator.ValidateAsync(_context).Wait();
-        }
-
-        private void ThenTheUserIsValidated()
-        {
-            _context.Result.IsError.ShouldBe(false);
-        }
-
-         private void ThenTheUserIsNotValidated()
-        {
-            _context.Result.IsError.ShouldBe(true);
-        }
-    }
-}

test/Ocelot.UnitTests/Controllers/FileConfigurationControllerTests.cs

@@ -1,14 +1,20 @@
+using System;
 using Microsoft.AspNetCore.Mvc;
 using Moq;
 using Ocelot.Configuration.File;
 using Ocelot.Configuration.Setter;
-using Ocelot.Controllers;
 using Ocelot.Errors;
 using Ocelot.Responses;
 using TestStack.BDDfy;
 using Xunit;
 using Shouldly;
 using Ocelot.Configuration.Provider;
+using Microsoft.Extensions.DependencyInjection;
+using Ocelot.Raft;
+using Rafty.Concensus;
+using Newtonsoft.Json;
+using Rafty.FiniteStateMachine;
+using Ocelot.Configuration;
 
 namespace Ocelot.UnitTests.Controllers
 {
@@ -19,18 +25,21 @@ namespace Ocelot.UnitTests.Controllers
         private Mock<IFileConfigurationSetter> _configSetter;
         private IActionResult _result;
         private FileConfiguration _fileConfiguration;
+        private Mock<IServiceProvider> _provider;
+        private Mock<INode> _node;
 
         public FileConfigurationControllerTests()
         {
+            _provider = new Mock<IServiceProvider>();
             _configGetter = new Mock<IFileConfigurationProvider>();
             _configSetter = new Mock<IFileConfigurationSetter>();
-            _controller = new FileConfigurationController(_configGetter.Object, _configSetter.Object);
+            _controller = new FileConfigurationController(_configGetter.Object, _configSetter.Object, _provider.Object);
         }
         
         [Fact]
         public void should_get_file_configuration()
         {
-            var expected = new OkResponse<FileConfiguration>(new FileConfiguration());
+            var expected = new Responses.OkResponse<FileConfiguration>(new FileConfiguration());
 
             this.Given(x => x.GivenTheGetConfigurationReturns(expected))
                 .When(x => x.WhenIGetTheFileConfiguration())
@@ -41,7 +50,7 @@ namespace Ocelot.UnitTests.Controllers
         [Fact]
         public void should_return_error_when_cannot_get_config()
         {
-            var expected = new ErrorResponse<FileConfiguration>(It.IsAny<Error>());
+            var expected = new Responses.ErrorResponse<FileConfiguration>(It.IsAny<Error>());
 
              this.Given(x => x.GivenTheGetConfigurationReturns(expected))
                 .When(x => x.WhenIGetTheFileConfiguration())
@@ -56,26 +65,81 @@ namespace Ocelot.UnitTests.Controllers
             var expected = new FileConfiguration();
 
             this.Given(x => GivenTheFileConfiguration(expected))
-                .And(x => GivenTheConfigSetterReturnsAnError(new OkResponse()))
+                .And(x => GivenTheConfigSetterReturns(new OkResponse()))
                 .When(x => WhenIPostTheFileConfiguration())
                 .Then(x => x.ThenTheConfigrationSetterIsCalledCorrectly())
                 .BDDfy();
         }
 
+        [Fact]
+        public void should_post_file_configuration_using_raft_node()
+        {
+            var expected = new FileConfiguration();
+
+            this.Given(x => GivenTheFileConfiguration(expected))
+                .And(x => GivenARaftNodeIsRegistered())
+                .And(x => GivenTheNodeReturnsOK())
+                .And(x => GivenTheConfigSetterReturns(new OkResponse()))
+                .When(x => WhenIPostTheFileConfiguration())
+                .Then(x => x.ThenTheNodeIsCalledCorrectly())
+                .BDDfy();
+        }
+
+        [Fact]
+        public void should_return_error_when_cannot_set_config_using_raft_node()
+        {
+            var expected = new FileConfiguration();
+
+            this.Given(x => GivenTheFileConfiguration(expected))
+                .And(x => GivenARaftNodeIsRegistered())
+                .And(x => GivenTheNodeReturnsError())
+                .When(x => WhenIPostTheFileConfiguration())
+                .Then(x => ThenTheResponseIs<BadRequestObjectResult>())
+                .BDDfy();
+        }
+
         [Fact]
         public void should_return_error_when_cannot_set_config()
         {
             var expected = new FileConfiguration();
 
             this.Given(x => GivenTheFileConfiguration(expected))
-                .And(x => GivenTheConfigSetterReturnsAnError(new ErrorResponse(new FakeError())))
+                .And(x => GivenTheConfigSetterReturns(new ErrorResponse(new FakeError())))
                 .When(x => WhenIPostTheFileConfiguration())
                 .Then(x => x.ThenTheConfigrationSetterIsCalledCorrectly())
                 .And(x => ThenTheResponseIs<BadRequestObjectResult>())
                 .BDDfy();
         }
 
-        private void GivenTheConfigSetterReturnsAnError(Response response)
+
+        private void ThenTheNodeIsCalledCorrectly()
+        {
+            _node.Verify(x => x.Accept(It.IsAny<UpdateFileConfiguration>()), Times.Once);
+        }
+
+        private void GivenARaftNodeIsRegistered()
+        {
+            _node = new Mock<INode>();
+            _provider
+                .Setup(x => x.GetService(typeof(INode)))
+                .Returns(_node.Object);
+        }
+
+        private void GivenTheNodeReturnsOK()
+        {
+            _node
+                .Setup(x => x.Accept(It.IsAny<UpdateFileConfiguration>()))
+                .Returns(new Rafty.Concensus.OkResponse<UpdateFileConfiguration>(new UpdateFileConfiguration(new FileConfiguration())));
+        }
+
+        private void GivenTheNodeReturnsError()
+        {
+            _node
+                .Setup(x => x.Accept(It.IsAny<UpdateFileConfiguration>()))
+                .Returns(new Rafty.Concensus.ErrorResponse<UpdateFileConfiguration>("error", new UpdateFileConfiguration(new FileConfiguration())));
+        }
+
+        private void GivenTheConfigSetterReturns(Response response)
         {
             _configSetter
                 .Setup(x => x.Set(It.IsAny<FileConfiguration>()))
@@ -103,7 +167,7 @@ namespace Ocelot.UnitTests.Controllers
            _result.ShouldBeOfType<T>();
         }
 
-        private void GivenTheGetConfigurationReturns(Response<FileConfiguration> fileConfiguration)
+        private void GivenTheGetConfigurationReturns(Ocelot.Responses.Response<FileConfiguration> fileConfiguration)
         {
             _configGetter
                 .Setup(x => x.Get())

test/Ocelot.UnitTests/Controllers/OutputCacheControllerTests.cs

@@ -1,10 +1,9 @@
 using Xunit;
 using Shouldly;
 using TestStack.BDDfy;
-using Ocelot.Controllers;
+using Ocelot.Cache;
 using System;
 using Moq;
-using Ocelot.Cache;
 using System.Net.Http;
 using System.Collections.Generic;
 using Microsoft.AspNetCore.Mvc;

test/Ocelot.UnitTests/DependencyInjection/OcelotBuilderTests.cs

@@ -75,6 +75,16 @@ namespace Ocelot.UnitTests.DependencyInjection
                 .BDDfy();
         }
 
+         [Fact]
+        public void should_set_up_rafty()
+        {            
+            this.Given(x => WhenISetUpOcelotServices())
+                .When(x => WhenISetUpRafty())
+                .Then(x => ThenAnExceptionIsntThrown())
+                .Then(x => ThenTheCorrectAdminPathIsRegitered())
+                .BDDfy();
+        }
+
         [Fact]
         public void should_use_logger_factory()
         {
@@ -85,6 +95,13 @@ namespace Ocelot.UnitTests.DependencyInjection
                 .BDDfy();
         }
 
+        private void ThenTheCorrectAdminPathIsRegitered()
+        {
+            _serviceProvider = _services.BuildServiceProvider();
+            var path = _serviceProvider.GetService<IAdministrationPath>();
+            path.Path.ShouldBe("/administration");
+        }
+
         private void OnlyOneVersionOfEachCacheIsRegistered()
         {
             var outputCache = _services.Single(x => x.ServiceType == typeof(IOcelotCache<CachedResponse>));
@@ -111,6 +128,18 @@ namespace Ocelot.UnitTests.DependencyInjection
             }       
         }
 
+        private void WhenISetUpRafty()
+        {
+            try
+            {
+                _ocelotBuilder.AddAdministration("/administration", "secret").AddRafty();
+            }
+            catch (Exception e)
+            {
+                _ex = e;
+            }       
+        }
+
         private void ThenAnOcelotBuilderIsReturned()
         {
             _ocelotBuilder.ShouldBeOfType<OcelotBuilder>();

test/Ocelot.UnitTests/Raft/OcelotFiniteStateMachineTests.cs

@@ -0,0 +1,45 @@
+using Moq;
+using Ocelot.Configuration.Setter;
+using Ocelot.Raft;
+using TestStack.BDDfy;
+using Xunit;
+
+namespace Ocelot.UnitTests.Raft
+{
+    public class OcelotFiniteStateMachineTests
+    {
+        private UpdateFileConfiguration _command;
+        private OcelotFiniteStateMachine _fsm;
+        private Mock<IFileConfigurationSetter> _setter;
+
+        public OcelotFiniteStateMachineTests()
+        {
+            _setter = new Mock<IFileConfigurationSetter>();
+            _fsm = new OcelotFiniteStateMachine(_setter.Object);
+        }
+
+        [Fact]
+        public void should_handle_update_file_configuration_command()
+        {
+            this.Given(x => GivenACommand(new UpdateFileConfiguration(new Ocelot.Configuration.File.FileConfiguration())))
+                .When(x => WhenTheCommandIsHandled())
+                .Then(x => ThenTheStateIsUpdated())
+                .BDDfy();
+        }
+
+        private void GivenACommand(UpdateFileConfiguration command)
+        {
+            _command = command;
+        }
+
+        private void WhenTheCommandIsHandled()
+        {
+             _fsm.Handle(new Rafty.Log.LogEntry(_command, _command.GetType(), 0));
+        }
+
+        private void ThenTheStateIsUpdated()
+        {
+            _setter.Verify(x => x.Set(_command.Configuration), Times.Once);
+        }
+    }
+}