Release/13.6.0 (#895)

* Fixed Format Issue for Kubernetes ServiceDiscoveryProvider

* Fixes broken links (#858)

* Fix link to issue 262

* Fixes broken link to issue 340

* Fixed broken link to issue 340 (#857)

* Update information for Okta Authorization (#853)

* +dynamic claim variables (#855)

incl. tests

* IOcelotPipelineBuilder.Use(): Return IOcelotPipelineBuilder (#875)

Fixes ThreeMammals/Ocelot#685

* Fix UpstreamHost checking when reroutes duplicate validation (#864)

* Format json in reame (#877)

Format json file in AdministrationApi ReadMe

* kubernetes use in cluster (#882)

* refactor :kubernetes use in cluster

* feat:delete KubeClient

* add more flexible method to config ocelot pipeline (#880)

* update k8s doc & samples (#885)

* refactor :kubernetes use in cluster

* feat:delete KubeClient

* feat :  update k8s doc & samples

* Update kubernetes.rst

* Fix/issue666 (#889)

* cache key now can generate from query string for request with Get Methods and request content for requests with post methods

* MD5Helper Added. OutputCacheMiddleware now can generate cache key using method, url and content

* unit test created for CacheKeyGenerator

* CacheKeyGenerator Registered in OcelotBuilder as singletone

* Fix issue #890 IDefinedAggregator can't handle error codes from downstream requests (#892)

* Release/13.2.0 (#834)

* Fix formatting in getting started page (#752)

* updated release docs (#745)

* Update README.md (#756)

Fixed typo "Ocleot"

* Fixed typo there => their (#763)

* Some Typo fixes (#765)

* Typo algorythm => algorithm (#764)

* Typo querystring => query string (#766)

* Typo usual => usually (#767)

* Typos (#768)

* kubernetes provider (#772)

* feat: Kubernetes ServiceDiscoveryProvider

* 编写k8s测试例子

* feat:fix kube config

* feat: remove port

* feat : complete the k8s test

* feat :  add kubeserviceDiscovery test

* feat : add kube provider unittest

* feat :add kubetnetes docs

how to use ocelot with kubetnetes docs

* keep the configuration as simple as possible, no qos, no cache

* fix: use http

* add PollingKubeServiceDiscovery

* feat : refactor logger

* feat : add  pollkube docs

* feat:Remove unnecessary code

* feat : code-block json

* fix issue #661 for Advanced aggregations (#704)

* Add Advanced Aggregation Feature

* fix overwrite error

* distinct data for better performance

* remove constructor parameter

* fix tests issue

* fix tests

* fix tests issue

* Add UnitTest and AcceptanceTest

* fix responseKeys typo

* Update SimpleJsonResponseAggregator.cs

* change port

* Fix code example for SSL Errors (#780)

DangerousAcceptAnyServerCertificateValidator has to be set to "true" to disable certification validation, not "false".

* Changed wording for ease of reading (#776)

Just some wording changes for clarification.

* Ignore response content if null (fix #785) (#786)

* fix bug #791 (#795)

* Update loadbalancer.rst (#796)

* UriBuilder - remove leading question mark #747 (#794)

* Update qualityofservice.rst (#801)

Tiny typo

* K8s package (#804)

* feat: Kubernetes ServiceDiscoveryProvider

* 编写k8s测试例子

* feat:fix kube config

* feat: remove port

* feat : complete the k8s test

* feat :  add kubeserviceDiscovery test

* feat : add kube provider unittest

* feat :add kubetnetes docs

how to use ocelot with kubetnetes docs

* keep the configuration as simple as possible, no qos, no cache

* fix: use http

* add PollingKubeServiceDiscovery

* feat : refactor logger

* feat : add  pollkube docs

* feat:Remove unnecessary code

* feat : code-block json

* feat: publish package Ocelot.Provider.Kubernetes

* Okta integration (#807)

Okta integration

* update cliamsParser (#798)

* update cliamsParser

* update using

* IOcelotBuilder opens the IMvcCoreBuilder property for easy customization (#790)

* IOcelotBuilder opens the IMvcCoreBuilder property for easy customization

* Adjustment code

* nuget package (#809)

* feat: Kubernetes ServiceDiscoveryProvider

* 编写k8s测试例子

* feat:fix kube config

* feat: remove port

* feat : complete the k8s test

* feat :  add kubeserviceDiscovery test

* feat : add kube provider unittest

* feat :add kubetnetes docs

how to use ocelot with kubetnetes docs

* keep the configuration as simple as possible, no qos, no cache

* fix: use http

* add PollingKubeServiceDiscovery

* feat : refactor logger

* feat : add  pollkube docs

* feat:Remove unnecessary code

* feat : code-block json

* feat: publish package Ocelot.Provider.Kubernetes

* feat : nuget package

* fix: Namesapce Spelling wrong

* fix:Namesapce Spelling Wrong

* Fix: errors when using rate limiting (#811)

* Fix: errors when using rate limiting
Add: QuotaExceededError class for requesting too much
Add: QuotaExceededError error code
Add: Add an error when limit is reached
Reflact: Extract GetResponseMessage method for getting default or configured response message for requ

* Fix: modify check_we_have_considered_all_errors_in_these_tests for adding a new OcelotErrorCode

* added missing COPY csproj files (#821)

* Add note on In-Process hosting (#816)

When using ASP.NET Core 2.2 with In-Process hosting in IIS it's important to use .UseIIS() instead of .UseIISIntegration().

* Fix bug: (#810)

If the registered Consul node is unexpectedly down and not restarted immediately, other services should continue to find the registered service.

* Fixed Dockerfile (missing Kubernetes)

* Revert "Fix bug: (#810)" (#823)

This reverts commit 19c80afb05.

* remove duplicate `IHttpRequester` register (#819)

* remove duplicate `IHttpRequester` register

* reserve the first

* fix HttpRequesterMiddleware does not call next bug (#830)

call next so that we can do something with the response, such as add some custom header etc...

* Removed Packing to fix issues, will be sorted out after create a nuget package on Nuget.Org (#831)

* Allows access to unpass node (#825)

* Fix bug:
If the registered Consul node is unexpectedly down and not restarted immediately, other services should continue to find the registered service.

* fix bug:
If the registered Consul node is unexpectedly down and not restarted immediately, other services should continue to find the registered service.

* Updated FluentValidations Nuget Package (#833)

* Removed Warnings

* Make the full DownstreamContext available to user defined aggregators

This allows error codes to be handled
This commit is contained in:
Thiago Loureiro
2019-05-20 16:25:44 +08:00
committed by GitHub
parent 5db449b000
commit 57580afa74
40 changed files with 383 additions and 128 deletions

View File

@ -33,6 +33,10 @@
builder.Services.RemoveAll(typeof(IOcelotCache<FileConfiguration>));
builder.Services.AddSingleton<ICacheManager<FileConfiguration>>(fileConfigCacheManagerOutputCache);
builder.Services.AddSingleton<IOcelotCache<FileConfiguration>>(fileConfigCacheManager);
builder.Services.RemoveAll(typeof(ICacheKeyGenerator));
builder.Services.AddSingleton<ICacheKeyGenerator, CacheKeyGenerator>();
return builder;
}
}

View File

@ -15,13 +15,14 @@ namespace Ocelot.Provider.Kubernetes
private IOcelotLogger logger;
private IKubeApiClient kubeApi;
public Kube(KubeRegistryConfiguration kubeRegistryConfiguration, IOcelotLoggerFactory factory, IKubeApiClientFactory kubeClientFactory)
public Kube(KubeRegistryConfiguration kubeRegistryConfiguration, IOcelotLoggerFactory factory, IKubeApiClient kubeApi)
{
this.kubeRegistryConfiguration = kubeRegistryConfiguration;
this.logger = factory.CreateLogger<Kube>();
this.kubeApi = kubeClientFactory.Get(kubeRegistryConfiguration);
this.kubeApi = kubeApi;
}
public async Task<List<Service>> Get()
{
var service = await kubeApi.ServicesV1()

View File

@ -5,16 +5,8 @@ namespace Ocelot.Provider.Kubernetes
{
public class KubeRegistryConfiguration
{
public Uri ApiEndPoint { get; set; }
public string KubeNamespace { get; set; }
public string KeyOfServiceInK8s { get; set; }
public KubeAuthStrategy AuthStrategy { get; set; }
public string AccessToken { get; set; }
public bool AllowInsecure { get; set; }
}
}

View File

@ -16,18 +16,14 @@ namespace Ocelot.Provider.Kubernetes
private static ServiceDiscovery.Providers.IServiceDiscoveryProvider GetkubeProvider(IServiceProvider provider, Configuration.ServiceProviderConfiguration config, string name, IOcelotLoggerFactory factory)
{
var kubeClientFactory = provider.GetService<IKubeApiClientFactory>();
var kubeClient = provider.GetService<IKubeApiClient>();
var k8sRegistryConfiguration = new KubeRegistryConfiguration()
{
ApiEndPoint = new Uri($"https://{config.Host}:{config.Port}"),
KeyOfServiceInK8s = name,
KubeNamespace = config.Namespace,
AuthStrategy = KubeAuthStrategy.BearerToken,
AccessToken = config.Token,
AllowInsecure = true // Don't validate server certificate
};
var k8sServiceDiscoveryProvider = new Kube(k8sRegistryConfiguration, factory, kubeClientFactory);
var k8sServiceDiscoveryProvider = new Kube(k8sRegistryConfiguration, factory, kubeClient);
if (config.Type?.ToLower() == "pollkube")
{
return new PollKube(config.PollingInterval, factory, k8sServiceDiscoveryProvider);

View File

@ -25,7 +25,13 @@
</PropertyGroup>
<ItemGroup>
<PackageReference Include="KubeClient" Version="2.2.4" />
<Compile Remove="IKubeApiClientFactory.cs" />
<Compile Remove="KubeApiClientFactory.cs" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="KubeClient" Version="2.2.11" />
<PackageReference Include="KubeClient.Extensions.DependencyInjection" Version="2.2.11" />
</ItemGroup>
<ItemGroup>

View File

@ -1,14 +1,15 @@
using Microsoft.Extensions.DependencyInjection;
using KubeClient;
using Microsoft.Extensions.DependencyInjection;
using Ocelot.DependencyInjection;
namespace Ocelot.Provider.Kubernetes
{
public static class OcelotBuilderExtensions
{
public static IOcelotBuilder AddKubernetes(this IOcelotBuilder builder)
public static IOcelotBuilder AddKubernetes(this IOcelotBuilder builder, bool usePodServiceAccount = true)
{
builder.Services.AddSingleton(KubernetesProviderFactory.Get);
builder.Services.AddSingleton<IKubeApiClientFactory, KubeApiClientFactory>();
builder.Services.AddKubeClient(usePodServiceAccount);
return builder;
}
}

View File

@ -1,6 +1,13 @@
using System.Collections.Generic;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Security.Claims;
using System.Text.RegularExpressions;
using Ocelot.DownstreamRouteFinder.UrlMatcher;
using Ocelot.Middleware;
using Ocelot.Responses;
using Ocelot.Values;
namespace Ocelot.Authorisation
{
@ -15,8 +22,11 @@ namespace Ocelot.Authorisation
_claimsParser = claimsParser;
}
public Response<bool> Authorise(ClaimsPrincipal claimsPrincipal, Dictionary<string, string> routeClaimsRequirement)
{
public Response<bool> Authorise(
ClaimsPrincipal claimsPrincipal,
Dictionary<string, string> routeClaimsRequirement,
List<PlaceholderNameAndValue> urlPathPlaceholderNameAndValues
){
foreach (var required in routeClaimsRequirement)
{
var values = _claimsParser.GetValuesByClaimType(claimsPrincipal.Claims, required.Key);
@ -27,12 +37,50 @@ namespace Ocelot.Authorisation
}
if (values.Data != null)
{
var authorised = values.Data.Contains(required.Value);
if (!authorised)
{
// dynamic claim
var match = Regex.Match(required.Value, @"^{(?<variable>.+)}$");
if (match.Success)
{
return new ErrorResponse<bool>(new ClaimValueNotAuthorisedError(
$"claim value: {string.Join(", ", values.Data)} is not the same as required value: {required.Value} for type: {required.Key}"));
var variableName = match.Captures[0].Value;
var matchingPlaceholders = urlPathPlaceholderNameAndValues.Where(p => p.Name.Equals(variableName)).Take(2).ToArray();
if (matchingPlaceholders.Length == 1)
{
// match
var actualValue = matchingPlaceholders[0].Value;
var authorised = values.Data.Contains(actualValue);
if (!authorised)
{
return new ErrorResponse<bool>(new ClaimValueNotAuthorisedError(
$"dynamic claim value for {variableName} of {string.Join(", ", values.Data)} is not the same as required value: {actualValue}"));
}
}
else
{
// config error
if (matchingPlaceholders.Length == 0)
{
return new ErrorResponse<bool>(new ClaimValueNotAuthorisedError(
$"config error: requires variable claim value: {variableName} placeholders does not contain that variable: {string.Join(", ", urlPathPlaceholderNameAndValues.Select(p=>p.Name))}"));
}
else
{
return new ErrorResponse<bool>(new ClaimValueNotAuthorisedError(
$"config error: requires variable claim value: {required.Value} but placeholders are ambiguous: {string.Join(", ", urlPathPlaceholderNameAndValues.Where(p=>p.Name.Equals(variableName)).Select(p => p.Value))}"));
}
}
}
else
{
// static claim
var authorised = values.Data.Contains(required.Value);
if (!authorised)
{
return new ErrorResponse<bool>(new ClaimValueNotAuthorisedError(
$"claim value: {string.Join(", ", values.Data)} is not the same as required value: {required.Value} for type: {required.Key}"));
}
}
}
else

View File

@ -1,5 +1,9 @@
using System.Security.Claims;
using Ocelot.Configuration;
using Ocelot.DownstreamRouteFinder.UrlMatcher;
using Ocelot.Responses;
using Ocelot.Values;
namespace Ocelot.Authorisation
{
@ -7,6 +11,10 @@ namespace Ocelot.Authorisation
public interface IClaimsAuthoriser
{
Response<bool> Authorise(ClaimsPrincipal claimsPrincipal, Dictionary<string, string> routeClaimsRequirement);
Response<bool> Authorise(
ClaimsPrincipal claimsPrincipal,
Dictionary<string, string> routeClaimsRequirement,
List<PlaceholderNameAndValue> urlPathPlaceholderNameAndValues
);
}
}

View File

@ -57,8 +57,8 @@
if (IsAuthorisedRoute(context.DownstreamReRoute))
{
Logger.LogInformation("route is authorised");
var authorised = _claimsAuthoriser.Authorise(context.HttpContext.User, context.DownstreamReRoute.RouteClaimsRequirement);
var authorised = _claimsAuthoriser.Authorise(context.HttpContext.User, context.DownstreamReRoute.RouteClaimsRequirement, context.TemplatePlaceholderNameAndValues);
if (authorised.IsError)
{

View File

@ -0,0 +1,19 @@
using System.Text;
using System.Threading.Tasks;
using Ocelot.Middleware;
namespace Ocelot.Cache {
public class CacheKeyGenerator : ICacheKeyGenerator {
public string GenerateRequestCacheKey(DownstreamContext context) {
string hashedContent = null;
StringBuilder downStreamUrlKeyBuilder = new StringBuilder($"{context.DownstreamRequest.Method}-{context.DownstreamRequest.OriginalString}");
if (context.DownstreamRequest.Content != null) {
string requestContentString = Task.Run(async () => await context.DownstreamRequest.Content.ReadAsStringAsync()).Result;
downStreamUrlKeyBuilder.Append(requestContentString);
}
hashedContent = MD5Helper.GenerateMd5(downStreamUrlKeyBuilder.ToString());
return hashedContent;
}
}
}

View File

@ -0,0 +1,7 @@
using Ocelot.Middleware;
namespace Ocelot.Cache {
public interface ICacheKeyGenerator {
string GenerateRequestCacheKey(DownstreamContext context);
}
}

View File

@ -0,0 +1,22 @@
using System.Security.Cryptography;
using System.Text;
namespace Ocelot.Cache {
public static class MD5Helper {
public static string GenerateMd5(byte[] contentBytes) {
MD5 md5 = MD5.Create();
byte[] hash = md5.ComputeHash(contentBytes);
StringBuilder sb = new StringBuilder();
for (int i = 0; i < hash.Length; i++) {
sb.Append(hash[i].ToString("X2"));
}
return sb.ToString();
}
public static string GenerateMd5(string contentString) {
byte[] contentBytes = Encoding.Unicode.GetBytes(contentString);
return GenerateMd5(contentBytes);
}
}
}

View File

@ -7,19 +7,23 @@
using Ocelot.Logging;
using Ocelot.Middleware;
using System.IO;
using System.Text;
public class OutputCacheMiddleware : OcelotMiddleware
{
private readonly OcelotRequestDelegate _next;
private readonly IOcelotCache<CachedResponse> _outputCache;
private readonly ICacheKeyGenerator _cacheGeneratot;
public OutputCacheMiddleware(OcelotRequestDelegate next,
IOcelotLoggerFactory loggerFactory,
IOcelotCache<CachedResponse> outputCache)
IOcelotCache<CachedResponse> outputCache,
ICacheKeyGenerator cacheGeneratot)
:base(loggerFactory.CreateLogger<OutputCacheMiddleware>())
{
_next = next;
_outputCache = outputCache;
_cacheGeneratot = cacheGeneratot;
}
public async Task Invoke(DownstreamContext context)
@ -31,10 +35,11 @@
}
var downstreamUrlKey = $"{context.DownstreamRequest.Method}-{context.DownstreamRequest.OriginalString}";
string downStreamRequestCacheKey = _cacheGeneratot.GenerateRequestCacheKey(context);
Logger.LogDebug($"Started checking cache for {downstreamUrlKey}");
var cached = _outputCache.Get(downstreamUrlKey, context.DownstreamReRoute.CacheOptions.Region);
var cached = _outputCache.Get(downStreamRequestCacheKey, context.DownstreamReRoute.CacheOptions.Region);
if (cached != null)
{
@ -61,12 +66,13 @@
cached = await CreateCachedResponse(context.DownstreamResponse);
_outputCache.Add(downstreamUrlKey, cached, TimeSpan.FromSeconds(context.DownstreamReRoute.CacheOptions.TtlSeconds), context.DownstreamReRoute.CacheOptions.Region);
_outputCache.Add(downStreamRequestCacheKey, cached, TimeSpan.FromSeconds(context.DownstreamReRoute.CacheOptions.TtlSeconds), context.DownstreamReRoute.CacheOptions.Region);
Logger.LogDebug($"finished response added to cache for {downstreamUrlKey}");
}
private void SetHttpResponseMessageThisRequest(DownstreamContext context, DownstreamResponse response)
private void SetHttpResponseMessageThisRequest(DownstreamContext context,
DownstreamResponse response)
{
context.DownstreamResponse = response;
}

View File

@ -122,7 +122,7 @@
{
var matchingReRoutes = reRoutes
.Where(r => r.UpstreamPathTemplate == reRoute.UpstreamPathTemplate
&& (r.UpstreamHost != reRoute.UpstreamHost || reRoute.UpstreamHost == null))
&& (r.UpstreamHost == reRoute.UpstreamHost || reRoute.UpstreamHost == null))
.ToList();
if (matchingReRoutes.Count == 1)

View File

@ -107,6 +107,7 @@ namespace Ocelot.DependencyInjection
Services.TryAddSingleton<IHttpHandlerOptionsCreator, HttpHandlerOptionsCreator>();
Services.TryAddSingleton<IDownstreamAddressesCreator, DownstreamAddressesCreator>();
Services.TryAddSingleton<IDelegatingHandlerHandlerFactory, DelegatingHandlerHandlerFactory>();
Services.TryAddSingleton<ICacheKeyGenerator, CacheKeyGenerator>();
// see this for why we register this as singleton http://stackoverflow.com/questions/37371264/invalidoperationexception-unable-to-resolve-service-for-type-microsoft-aspnetc
// could maybe use a scoped data repository

View File

@ -5,6 +5,6 @@ namespace Ocelot.Middleware.Multiplexer
{
public interface IDefinedAggregator
{
Task<DownstreamResponse> Aggregate(List<DownstreamResponse> responses);
Task<DownstreamResponse> Aggregate(List<DownstreamContext> responses);
}
}

View File

@ -1,5 +1,4 @@
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Ocelot.Configuration;
@ -21,7 +20,7 @@ namespace Ocelot.Middleware.Multiplexer
if (!aggregator.IsError)
{
var aggregateResponse = await aggregator.Data
.Aggregate(downstreamResponses.Select(x => x.DownstreamResponse).ToList());
.Aggregate(downstreamResponses);
originalContext.DownstreamResponse = aggregateResponse;
}

View File

@ -1,22 +1,22 @@
namespace Ocelot.Middleware
{
using System;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.Options;
using System.Diagnostics;
using DependencyInjection;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Options;
using Ocelot.Configuration;
using Ocelot.Configuration.Creator;
using Ocelot.Configuration.File;
using Ocelot.Configuration.Repository;
using Ocelot.Configuration.Setter;
using Ocelot.Responses;
using Ocelot.Logging;
using Ocelot.Middleware.Pipeline;
using Microsoft.Extensions.DependencyInjection;
using Ocelot.Responses;
using System;
using System.Diagnostics;
using System.Linq;
using System.Threading.Tasks;
public static class OcelotMiddlewareExtensions
{
@ -42,6 +42,30 @@
return CreateOcelotPipeline(builder, pipelineConfiguration);
}
public static Task<IApplicationBuilder> UseOcelot(this IApplicationBuilder app, Action<IOcelotPipelineBuilder, OcelotPipelineConfiguration> builderAction)
=> UseOcelot(app, builderAction, new OcelotPipelineConfiguration());
public static async Task<IApplicationBuilder> UseOcelot(this IApplicationBuilder app, Action<IOcelotPipelineBuilder, OcelotPipelineConfiguration> builderAction, OcelotPipelineConfiguration configuration)
{
await CreateConfiguration(app); // initConfiguration
ConfigureDiagnosticListener(app);
var ocelotPipelineBuilder = new OcelotPipelineBuilder(app.ApplicationServices);
builderAction?.Invoke(ocelotPipelineBuilder, configuration ?? new OcelotPipelineConfiguration());
var ocelotDelegate = ocelotPipelineBuilder.Build();
app.Properties["analysis.NextMiddlewareName"] = "TransitionToOcelotMiddleware";
app.Use(async (context, task) =>
{
var downstreamContext = new DownstreamContext(context);
await ocelotDelegate.Invoke(downstreamContext);
});
return app;
}
private static IApplicationBuilder CreateOcelotPipeline(IApplicationBuilder builder, OcelotPipelineConfiguration pipelineConfiguration)
{
var pipelineBuilder = new OcelotPipelineBuilder(builder.ApplicationServices);

View File

@ -9,7 +9,7 @@ namespace Ocelot.Middleware.Pipeline
public interface IOcelotPipelineBuilder
{
IServiceProvider ApplicationServices { get; }
OcelotPipelineBuilder Use(Func<OcelotRequestDelegate, OcelotRequestDelegate> middleware);
IOcelotPipelineBuilder Use(Func<OcelotRequestDelegate, OcelotRequestDelegate> middleware);
OcelotRequestDelegate Build();
IOcelotPipelineBuilder New();
}

View File

@ -27,7 +27,7 @@ namespace Ocelot.Middleware.Pipeline
public IServiceProvider ApplicationServices { get; }
public OcelotPipelineBuilder Use(Func<OcelotRequestDelegate, OcelotRequestDelegate> middleware)
public IOcelotPipelineBuilder Use(Func<OcelotRequestDelegate, OcelotRequestDelegate> middleware)
{
_middlewares.Add(middleware);
return this;

View File

@ -19,6 +19,7 @@ namespace Ocelot.Request.Middleware
Headers = _request.Headers;
AbsolutePath = _request.RequestUri.AbsolutePath;
Query = _request.RequestUri.Query;
Content = _request.Content;
}
public HttpRequestHeaders Headers { get; }
@ -37,6 +38,8 @@ namespace Ocelot.Request.Middleware
public string Query { get; set; }
public HttpContent Content { get; set; }
public HttpRequestMessage ToHttpRequestMessage()
{
var uriBuilder = new UriBuilder