How to use Memory Cache in AuthorizeFilter like Controller?

I have a global AuthorizeFilter as the following:

public class MyAuthorizationFilter : AuthorizeFilter { 

     public MyAuthorizationFilter(Authorization policy) : base(policy) { }

     public override Task OnAuthorizationAsync(AuthorizationFilterContext context) { ... }

}

And what I have done in Startup.cs:

services.AddMvc(x => {
     x.Filters.Add(new MyAuthorizationFilter(new AuthorizationPolicyBuilder().RequireAuthenticateUser().Build())); 
});

And I have added the service too

services.AddMemoryCache();

It seems the filter doesn't support DI that the way controller does.

Thanks.