[OOTB-infra] Virtual Hackathon - Capability of hosting integration/demo environments?

[Heiko Robert]

Hi Axel,

this wouldn't be too hard to manage. One question would be how to avoid
abuse / missusage. The owner of the servers and the domain will be
responsable for any damage if they can't proof the real actor!

Today cloning / setting up is a manual job taking ~ 1 hour.
Bottleneck would be DNS (INFRA team has no access to DNS)

Everything could be easily automated to create a running VM in minutes
except DNS (this could be fixed / worked around by a separate wildcard
domain):
* instanciating a new VM can be automated by ovatool command line
running anywhere having access to the vm network
* same for management (start/stop/kill/delete/monitor)
* where do you see any licensing issue? all tools are public available
or open source.
* simple HTTP(S) automation is easy since we already have only one
single reverse proxy (apache-proxy) with an makro to allow new targets
to be configured with one line of config
* simple firewall config (for ssh access) should be discussed (see below)

TODO:
* define master VM for cloning / save OVA as master
* create script for cloning (e.g. createnewVM source target)
* create script for loading different alfresco version (e.g. any subrelease)
* create script for vm network config (easiest would be static config
and default image will start with dhcp and hostname 'imagename')
* create script for adding virtual host to reverse proxy

At the end only one script/command should control all required use cases
(create/remove/stop/start/status/list).

Another aproach would be to configure only one VM to be set up with
containers / OpenVZ / vserver running in a separate subnet but I couln't
assist in this because this would consume much more work/time to get
this done automated.

Open:
* What would be an an adequate console access?
** Real names / registered, verified users should be required to get ssh
access
** all testing / public VMs should run in a separate subnet with no
connection to the OOTB production systems inside the VM network
** either public key auth or VPN should be forced. I would prefer VPN
because then we don't need to configre the firewall for any new vm. We
use pfsense. For now there is only UPnP or NAT-PMP as remote API
available. REST is announced for next major release (3.0) to automate
new firewall rules in case we don't go the vpn route. Anyway to
_automate_ VPN / firewall config we need either to create a curl based
script à la alf-shell-tool or wait for version 3.0. On the other hand if
a participant has only to be created once this may be acceptable to
create them by hand.

Heiko


Am 12.07.2016 um 17:57 schrieb Axel Faust:
> Hello INFRA,
> 
> During the BeeCon hackathon and afterwards various members of the Order
> (incl Richard) discussed continueing with the tradition of doing global,
> virtual hackathons in the Alfresco community. We tentatively have agreed
> to aim for a Friday within September to have the hackathon and that it
> would be an Alfresco event (in terms of announcements / general
> organisation). But that doesn't exclude us from thinking about options
> for the Order to support such an event, and the community members that
> participate.
> 
>  
> 
> One idea I had revolves around providing (short-term / temporary)
> hosting of integration or demo environments for hackathon project teams.
> Apart from potentially simplifying the setup / startup effort for new
> members of the community that participate, having a publicly accessible
> integration / demo environment for a hackathon project can also make it
> easier for other teams as well as Alfresco / Order "hackathon support
> staff" to check a project's progress; or act as a tester and provide
> suggestions for potential improvements.
> 
>  
> 
> Now, before I spend any more time fantasizing about project build /
> server restart automation, the more immediate question on my mind is:
> 
> *Could we as the Order with our current infrastructure support such a
> use case?*
> 
> Forget for a moment the financial issues that obviously will need to be
> addressed at some time as well. Can we (with our current infratstructure
> and processes):
> 
>  
> 
>   * Add VM(s) or host(s) to our domain within a few days (or with some
>     kind of self-service for a small set of authorised members in a few
>     hours) - same for tear-down after the event
>   * Support automation of instance setup / tear down, e.g. via command
>     line tooling either by agents on a host or via SSH
>   * Use open technologies / tools to avoid licensing issues when an
>     arbitrary number of users (from different backgrounds) use an
>     environment we temporarily provide / host
>   * Support simple update to any firewall configuration to forward
>     (dynamic) ports to the environments
>   * Support simple update to any proxy configuration to provide a
>     central HTTP(S) entry point
> 
> ?
> 
>  
> 
> A potential setup in my mind at this time is the following:
> 
>   * a separate host or resource-constrained VM is used to bundle all
>     demo environment
> 
> (a decent amount of memory, CPU and I/O needs to be provisioned, which
> is why I tend towards a separate host)
> 
>   * Within that host / VM, container-based virtualization is used to
>     provide standardized installs for the environments
>   * Each environment is publicly accessible when using HTTP(S)-based
>     protocols using a host name like
>     <projectName>.hackathon.orderofthebee.org
>     <http://hackathon.orderofthebee.org> (ideally we use only HTTPS
>     which a proxy terminates using a common certificate of the Order,
>     and internally use HTTP/AJP)
>   * A port forward for SSH should be done for each environment (if the
>     environment will actually provide SSH, which a container might not)
>   * If host / VM is not publicly accessibe itself, a port forward for
>     SSH needs to be done
>   * Host / VM polls GitHub of defined project for an environment, builds
>     project (Maven-based) and stops/starts container with updated WARs
>   * Automation is in place to quickly (re-)initialize an environment as
>     project teams are formed / when a data reset is required (automation
>     would likely need to update DNS / host or VM firewall settings)
> 
> I would exclude most (if not all) of the trickier protocols / Alfresco
> interfaces for the time being, and only consider HTTP-based access.
> 
>  
> 
> Even if we don't end up doing something in that direction for the
> hackathon (for cost/time/resource reasons), I would like to see us able
> to dynamically provide short-term integration / demo environments some
> day in the (near?) future, e.g. to support similar needs for conferences
> / local meetups. So if we can't currently provide this for technical /
> conceptual reasons or limitations, I hope we can derive a strategy for
> (or at least a list of) items that we woulud need to address.
> 
>  
> 
> So, what do you think? Are we currently in a situation that we could do
> something like this? And without excessive effort in terms of
> coordination / work, or too much reliance on the availability of a
> single or a few people?
> 
>  
> 
> Again, this is just an idea of mine at the moment that I would like some
> input / discussion on. This has not yet been discussed on the board or
> with Richard / Ole, as it may be irrelevant if you tell me that I am
> "out of my mind"...
> 
>  
> 
> Regards
> 
> Axel
> 
>  
> 
> 
> 
> _______________________________________________
> OOTB-infra mailing list
> OOTB-infra at xtreamlab.net
> http://www.xtreamlab.net/mailman/listinfo/ootb-infra
>