Store CDN entry for each library as configurable/alterable "entity"

[kreynen]

I've read #3, but I'm not sure I understand where this is headed.

There are so many use cases where auto detecting a library and trying to find it in various CDNs in the order you've hard coded in a module isn't going to work.

• The library is patched/altered
• The library is found in CDN, but that CDN's performance sucks for your region
• You want to a use a new CDN that isn't supported

It looks like you already have the auto discovery of libraries and finding them in various CDNs working... which is GREAT! What I've like to see is both the list of CDNs and libraries be stored as a configurable/alterable "entity". Using a full blown node based content type is overkill for this task. Entities are also probably more than you need. Defining a Ctools plugin for this seems like a good fit for the libraries. That would give you forms that users could use to alter the configuration as well Feature exports "out of the box".

Take a look at how this is defined in https://www.drupal.org/project/css_injector or https://www.drupal.org/project/js_injector.

For the supported CDNs, it would be create if these were added via a hook that allowed other modules to extend the list by defining the path, name, and version. Simple weight and status variables for each CDN would allow users to order which CDN is checked first. Each library would need to keep an ordered array of the CDN to use (if any) for that library.

That change would also allow sites to register private CDNs first with several public CDNs as fallbacks.

A final feature related to these changes is the ability to alter the CDN used. The use case for all of this could a university that runs a CDN on their network for on campus requests, but would use an external CDN to improve load times for external requests. I would like to be able to write a helper module for this that checks to see if the user is authenticated and the IP maing the request is on campus. If it is, use our internal CDN. If the user isn't authenticated and/or the request is from off campus, use the external CDN.

Does that make sense?

[mmilutinovic013]

Hi @kreynen just read through this and it makes sense. I'll write a thorough response about the plan for this module based on your comments tomorrow morning. Thanks for taking the time to do this and for the helpful feedback!

[mmilutinovic013]

@kreynen here is possibly a clearer explanation of where I would like this module to go:

As you mentioned -- scanning the libraries directory is not always the most effective way of loading files remotely. Patched files and availability of files on specific CDNs are always going to be an issue.

Because of this, I agree with you that the priority should be to make customizable CDN options so that the administrators can have a rule based plan on how they load their libraries. An example of how I imagine this looking would be similar to how the AngularJS module gives specific options for how you want to load their files (both locally and remotely). I liked your university use-case example and agree that this use case is one that I want to include as we can use this with ELMS:LN as well. Expanding on your IP example, I would like to have IP detection setup so that if a user is taking a course in China and a CDN is blocked in that country, the module can detect this and serve from either an accepted CDN, or locally.

Additionally, I would like to provide an option for hash verification of the files on the CDN to make sure that the CDN that is selected is not poisoned. This would be very important if the admins are defining their own CDNs or if the fallback is poisoned, as upon an invalid response, the module can load locally.

I would also like to allow users to have the option to download different versions of libraries and files from the CDN to their local install. This might be tricky, as my plan was to have this module write / update a module that defines the files in a drupal libraries api context. I'm looking for input whether this is a good idea or not.

I hope this is a clearer explanation of the intended direction of the module. @btopro may also have some additional thoughts on features that I may have forgotten to list. Here is the link to the issue in the ELMS:LN issue queue that prompted this module creation. (https://github.com/elmsln/elmsln/issues/994).

Please let me know if you have any additional questions -- and we can talk more here or on slack.

Thanks!
Mark Milutinovic
(mmilutinovic1313)

[btopro]

I don't think abstraction to support other CDN sources would be too difficult. It would be nice to have some kind of fallback mechanism like use A (might work in china), then B (works in china), then C (local).

[kreynen]

A (might work in china), then B (works in china), then C (local)

Wouldn't that require being able to classify the IPs making the requests into regions and apply rules to regions? and I'm assuming you'd also want to add custom IP definitions as well?

This is quickly going from easycdn to notsoeasycdn.

These aren't bad ideas, but I'm not sure the CMS is really the right place to do something this complicated. If we were going to do this at CU, we'd likely go a completely different route so we weren't dealing with the overhead Drupal adds to determine which url to give the user. We'd probably create an asset library of all the js and css assets we want to host outside of the servers hosting Drupal and configure the IP/region redirects at that level. So all requests from Drupal still point at a server within the colorado.edu domain, but that server decides whether to give the user the locally hosted asset or redirect them to a different server for the asset.

What I was hoping to get from easycdn is the option of writing a custom module that would apply a single rule based on internal/external requests. I'm not really requesting changes to the features you were planning to support "out of the box", but changes to how you are storing data so that we can alter it for our use case.

[btopro]

"writing a rule" != easy; goal is 0 config library addition and delivery :)

Also possible to async load CDN libraries and if they timeout based on user region connection then deliver another one. IP analysis is also possible ahead of time with reasonable accuracy in stuff like https://www.drupal.org/project/geoip_redirect to do some initial fallback support, realizing we'd have to have client side fallback. We need to write client side fallback for countries that block youtube when we try and deliver from youtube, then fallback to our own internal sources on-site to avoid people saying our media is blocked in their region. This goes beyond China as well since plugins like disconnect me can block CDNs for tracking purposes as well as company firewalls / VPNs which we get the occasional corporate enrole-y that has this issue as well.