[OOTB-hive] Docker(/Kubernetes) - My ideas for potential work (re: Daren's mail)

Fri Dec 7 14:39:42 GMT 2018

I appreciate the path you guys are aiming at. It is pretty much what I do
these days when working on the repo.

cheers
Andreas

On Fri, Dec 7, 2018 at 2:27 PM Ole Hejlskov <ole at phpfreak.dk> wrote:

> Hi all
>
> Forgive me, this will be a wall-of-text long email. TL;DR at the bottom
> for the lazy.
>
> First of all, I just wanted to recognise the impact the lack of a
> compatible SDK release has for all of you.
> Initially when we released SDK 3.0, it came with a bunch of promises of
> compatibility. However, this was all before we went into the world of
> microservices and containers. ACS6 has brought a lot of (good) changes, but
> unfortunately it has resulted in unfortunate side effects for the SDK.
>
> I am happy to say that we (Alfresco) *are* committed to bring a compatible
> SDK. We are working out the details on exactly when this might happen, and
> we hope to share more details soon. There.. I said it. "Soon". I flat out
> know how this will rub people the wrong way, but that is honestly the best
> I can do.
>
> OK with all of that out of the way, let's talk details.
>
> We made good progress during the hackathon, and lots of discussions later
> the approach we took was to spin up the related services (database and
> search services) as Docker containers.
>
> Now, why did we do this?
> Several reasons:
> 1) Solr isn't a WAR file anymore, it's an executable. We *can* have Maven
> spin up an external executable (and stop it again) as part of the build
> lifecycle, but getting this to work across multiple platforms will be a
> nightmare, if not straight out impossible, unless we leverage profiles (and
> thereby bring back the dreaded parent pom), which will be hard to maintain.
> In SDK 3.0 we wrapped all the craziness of dealing with Solr packaging into
> the Alfresco Maven Plugin. By externalising this into a docker container we
> removed A LOT of complexity in the plugin.
> Solution? Spin it up externally manually, or leverage Docker to automate
> it.
>
> 2) No more H2.
> Since ACS no longer leverages Hibernate, we can't control the dialect
> which enabled us to use H2. There is no way to cheat or work around it,
> can't be done as is today. ACS will require a database to be running.
> Solution? Spin it up externally manually, or leverage Docker to automate
> it.
>
>
> OK, so during the hackathon we relied on using the Tomcat Maven Plugin to
> start the repository itself. This worked at the time, but recent tests have
> discovered some very, very annoying issues:
> a) The Tomcat plugin has not been updated in years. I believe around 5
> years.
> b) The Tomcat plugin is stuck on Tomcat7
> c) The default version shipped with the Tomcat plugin will not load JARs
> compiled with Java9 (throwing exceptions about module-info.class and
> stuff). Latest versions of ACS Enterprise relies on some artifacts that
> contains a module-info.class, so exceptions are all over the place. Without
> having tested it, I believe the latest Community release will have the same
> issue
> d) You can, in theory, force the Tomcat plugin to use a different version,
> however you will be limited to Tomcat7 since there are incompatibilities.
> For the last 5 years this issue has been open (
> https://issues.apache.org/jira/browse/MTOMCAT-234) which would bring
> support for Tomcat8, but no resolution or progress made in years
>
> If anyone questions this, I'd be happy to dig up my notes and search
> histories, there are A LOT of issues with the plugin due it's been
> neglected for 5 years.
>
> OK, so a lot of issues with the Tomcat plugin. If we bring out the big
> hammer and force the Tomcat plugin to the latest Tomcat7 release, it will
> actually start without exceptions... if you are on Java8. Anything higher
> and it completely breaks apart.. you see where this is going.
>
> This is pretty much where things grinds to a halt.
>
> I no longer believe using an embedded Tomcat like this is viable.
>
> I would be willing to do experiments using the Cargo plugin, but given the
> direction we will only start to see more smaller services, Transformation
> service and event gateway just to name a few. It's not impossible to
> configure all of this without Docker, but it will be very complex and
> extremely difficult to maintain. (Remember how bad upgrading the between
> SDK 2.x releases?)
>
> The current design goals is to also leverage Docker to spin up ACS to do
> integration tests. Ultimately the SDK will consist of an archetype that can
> compile a JAR and AMP, then leverage Docker to orchestrate the integration
> tests.
>
> If you do not wish to leverage Docker, there is nothing stopping you from
> removing the Docker orchestration from the pom in your project and roll a
> custom setup. This has always been the goal: The archetypes serves as an
> example that gets you going quickly, but provide enough flexibility to make
> changes and only use the bits you need for your project.
>
> I assume the next logical question will be how this affects RAD and the
> overall development workflow. Spinning up ACS as a Docker container sounds
> like it will be painfully slow, and you have to wait for restarts on every
> change.
>
> Well, there are different approaches to be taken. We can mount the target
> directory into the container. We can leverage some stuff from JRebel, or
> look at how DECVM/Spring Loaded/HotSwap Agent are doing today. I have not
> personally done experiments in this area yet, so I'd love to hear feedback
> from people who have experimented with it.
>
> Again, please keep in mind that the archetypes can be tweaked and changed
> based on your needs. Some people loathe Docker. Some people hate using the
> embedded Tomcat. The archetypes are meant to give you a good starting point
> and you can totally tweak it to meet your needs.
>
> As always, we are very open to feedback. I will find time to commit my
> latest experiments (leveraging Tomcat plugin) to the sdk-4.0 branch ASAP.
>
> TL;DR:
> There will be a compatible SDK with ACS 6.0 and 6.1.
> The SDK will leverage Docker to spin up services such as Postgres and
> Search Services, and ultimately the repository itself.
>
> /Ole
>
> On 7 Dec 2018, at 11.14, Douglas C. R. Paes <douglascrp at gmail.com> wrote:
>
> Hello Jeff.
>
> The idea is not to force the Docker use, but to use it to make things
> easier. This is why Docker is used, to start PostgreSQL and Search Services
> only.
> For Alfresco and Share, there is nothing different, as the war files are
> used as always.
> But those are optional, and one can have then running locally using
> traditional methods.
>
>
> Douglas C. R. Paes
>
> *"D**one is better than perfect**"*
>
>
> Em qui, 6 de dez de 2018 às 23:45, Jeff Potts <jeffpotts01 at gmail.com>
> escreveu:
>
>> Why would I want the SDK to use a Docker image?
>>
>> My opinion is that it should continue to use the WAR as it has
>> previously. Otherwise, as you point out, the developer has to add
>> technology they may not have a need for.
>>
>> Jeff
>>
>> On Thu, Dec 6, 2018 at 11:36 AM Douglas C. R. Paes <douglascrp at gmail.com>
>> wrote:
>>
>>> Hello everyone.
>>>
>>> Because of the lack of support from Alfresco in the SDK space, and
>>> because I want to start using the Alfresco 6 version, I decided to check on
>>> the current status of the SDK project, and I took the work started by Ole
>>> on the last Hackathon, and tried to make it work.
>>>
>>> The result of my effort so far is available at
>>> https://github.com/douglascrp/alfresco-sdk/tree/sdk-4.0
>>>
>>> Important... I have been working only with
>>> *alfresco-platform-jar-archetype*, as I usually don't use the AIO
>>> archetype for anything.
>>>
>>> When you create the project using the alfresco-platform-jar-archetype,
>>> the project will be configured to start both PostgreSQL and Alfresco Search
>>> Service as Docker containers, but the project itself will be executed using
>>> the tomcat plugin, just like it was done in the SDK version 3.0.1.
>>>
>>> As this relies on the official Alfresco Docker image for Alfresco Search
>>> Service, anyone running it will have to perform the configuration described
>>> by Angel at
>>> https://github.com/keensoft/alfresco-docker-template/tree/master/templates/201806-GA#how-to-use-this-composition,
>>> which is to set the owner of the .data/solr-data.
>>> On Ubuntu: "sudo chown -R 1000:1000 data/solr-data"
>>>
>>> There are still some problems with activiti, and because of that, I had
>>> to configure the engline to be disabled by default.
>>>
>>> Feel free to test it and tell me what you think.
>>>
>>> Douglas C. R. Paes
>>>
>>> *"D**one is better than perfect**"*
>>>
>>>
>>> Em seg, 26 de nov de 2018 às 10:16, Angel Borroy <
>>> angel.borroy at keensoft.es> escreveu:
>>>
>>>> You are right, Axel.
>>>>
>>>> Sorry for trolling your initial (and great) initiative.
>>>>
>>>> Hoping people will fallback to main subject of this thread...
>>>>
>>>> Angel Borroy
>>>> keen*soft *a *UST **Global *company
>>>>
>>>> CONFIDENCIALIDAD:
>>>>
>>>> La presente comunicación y, en su caso, los ficheros que lleve
>>>> adjuntos, pertenecen exclusivamente a las personas a las que va dirigido y
>>>> puede contener información confidencial. Si usted no es el destinatario de
>>>> este mensaje (o la persona responsable de su entrega), considérese
>>>> advertido de que lo ha recibido por error y que cualquier uso, difusión,
>>>> reenvío o copia están prohibidos legalmente. Si ha recibido este mensaje
>>>> por error, por favor notifíquelo al remitente y proceda a destruirlo
>>>> inmediatamente.
>>>>
>>>> This message and the attached documents may contain
>>>> privileged/confidential information and intended solely for the use of the
>>>> individual to whom it is addressed. If you are not the intended recipient
>>>> (or responsible for delivery of the message to such a person) be advised
>>>> that you have received this message in error and that any use,
>>>> dissemination, forwarding, printing or copying of this e-mail is strictly
>>>> prohibited. If you have received this message in error please notify it to
>>>> the sender and destroy it immediately.
>>>>
>>>> On 26 November 2018 at 13:14:33, Axel Faust (
>>>> axel.faust.g at googlemail.com) wrote:
>>>>
>>>> This is only using a test / RC (release candidate) state of 6.1 EE - if
>>>> you look in artifacts.alfresco.com (I can via one of my customers)
>>>> you'll see there is no EE GA yet.
>>>>
>>>> Haven't we all learned by now not to trust Alfresco marketing (e.g.
>>>> what's written on the homepage) when it comes to 100% correctly /
>>>> transparently representing facts? CE vs EE comparison has been (naturally)
>>>> biased forever, and since Alfresco was planning a marketing push for
>>>> November it is also understandable that they might want to highlight the
>>>> next release, even if it may not yet be fully ready...
>>>>
>>>> Of course as always, advance / roadmap communication by Alfresco could
>>>> be better to avoid any such confusion.
>>>> On 26/11/2018 12:56, Angel Borroy wrote:
>>>>
>>>> At least, in my case, comes from here:
>>>>
>>>> <part1.00570E7E.7B019451 at googlemail>
>>>>
>>>> This looks like a public ACS EE 6.1 release...
>>>>
>>>> Angel Borroy
>>>> keen*soft *a *UST **Global *company
>>>>
>>>> email:   angel.borroy at keensoft.es
>>>> web:     http://www.keensoft.es
>>>> móvil:  +34 655 47 47 55 <//+34%20655%2047%2047%2055>
>>>>
>>>> CONFIDENCIALIDAD:
>>>>
>>>> La presente comunicación y, en su caso, los ficheros que lleve
>>>> adjuntos, pertenecen exclusivamente a las personas a las que va dirigido y
>>>> puede contener información confidencial. Si usted no es el destinatario de
>>>> este mensaje (o la persona responsable de su entrega), considérese
>>>> advertido de que lo ha recibido por error y que cualquier uso, difusión,
>>>> reenvío o copia están prohibidos legalmente. Si ha recibido este mensaje
>>>> por error, por favor notifíquelo al remitente y proceda a destruirlo
>>>> inmediatamente.
>>>>
>>>> This message and the attached documents may contain
>>>> privileged/confidential information and intended solely for the use of the
>>>> individual to whom it is addressed. If you are not the intended recipient
>>>> (or responsible for delivery of the message to such a person) be advised
>>>> that you have received this message in error and that any use,
>>>> dissemination, forwarding, printing or copying of this e-mail is strictly
>>>> prohibited. If you have received this message in error please notify it to
>>>> the sender and destroy it immediately.
>>>>
>>>> On 26 November 2018 at 12:54:48, Ole Hejlskov (ole at phpfreak.dk) wrote:
>>>>
>>>> Hi all
>>>>
>>>> I’m not sure where the confusion started, but I’ve just verified
>>>> internally that ACS 6.1 has entered the regression testing phase. Once this
>>>> completes there will be a ACS 6.1 GA release for Community, followed by 6.1
>>>> EE. The only thing that isn’t clear to me at this stage, is wether 6.1 EE
>>>> comes before 6.1 CE (GA) or vice-versa.
>>>>
>>>> Hope this clears things up.
>>>>
>>>> /Ole
>>>>
>>>> On 26 Nov 2018, at 11.33, Francesco Corti <fcorti at gmail.com> wrote:
>>>>
>>>> Good morning guys,
>>>>
>>>> I'm afraid there is something to adjust.
>>>> ACS 6.1 Community Edition won't be a GA release.
>>>> I think John mentioned the EE release (that will be GA by default).
>>>> Just a heads up.
>>>> In every case, the initiative continue to be a good idea.
>>>>
>>>> Cheers,
>>>>
>>>> -F
>>>>
>>>>
>>>>
>>>> Il giorno lun 26 nov 2018 alle ore 10:29 Angel Borroy <
>>>> angel.borroy at keensoft.es> ha scritto:
>>>>
>>>>> Hi,
>>>>>
>>>>> Sorry for being so vague. I have no more information than you, as UST
>>>>> Global is not still an Alfresco partner. We are applying from June but it
>>>>> looks like Alfresco has plenty of skilled partners in Spain and there is no
>>>>> room for one more. Two weeks ago I started an Enterprise project in
>>>>> collaboration with a local partner, as Alfresco forbid UST Global to work
>>>>> with Community (despite we are not partners nowadays). I’m starting to be
>>>>> very concerned about Alfresco Sales ecosystem, but probably it’s only my
>>>>> pov.
>>>>>
>>>>> Anyway, you are right, John Knowles said that they will publish an ACS
>>>>> Community release in GA status. But, afaik, this is the first Community
>>>>> release released after the Enterprise one. Error or strategy? Who knows…
>>>>>
>>>>> Let’s focus on this initiative and let’s keep apart all my bad
>>>>> feelings about the life in general.
>>>>>
>>>>> Best,
>>>>>
>>>>> Angel Borroy
>>>>> keen*soft *a *UST **Global *company
>>>>>
>>>>> email:   angel.borroy at keensoft.es
>>>>> web:     http://www.keensoft.es
>>>>> móvil:  +34 655 47 47 55
>>>>>
>>>>> CONFIDENCIALIDAD:
>>>>>
>>>>> La presente comunicación y, en su caso, los ficheros que lleve
>>>>> adjuntos, pertenecen exclusivamente a las personas a las que va dirigido y
>>>>> puede contener información confidencial. Si usted no es el destinatario de
>>>>> este mensaje (o la persona responsable de su entrega), considérese
>>>>> advertido de que lo ha recibido por error y que cualquier uso, difusión,
>>>>> reenvío o copia están prohibidos legalmente. Si ha recibido este mensaje
>>>>> por error, por favor notifíquelo al remitente y proceda a destruirlo
>>>>> inmediatamente.
>>>>>
>>>>> This message and the attached documents may contain
>>>>> privileged/confidential information and intended solely for the use of the
>>>>> individual to whom it is addressed. If you are not the intended recipient
>>>>> (or responsible for delivery of the message to such a person) be advised
>>>>> that you have received this message in error and that any use,
>>>>> dissemination, forwarding, printing or copying of this e-mail is strictly
>>>>> prohibited. If you have received this message in error please notify it to
>>>>> the sender and destroy it immediately.
>>>>>
>>>>> On 26 November 2018 at 09:43:45, Axel Faust (
>>>>> axel.faust.g at googlemail.com) wrote:
>>>>>
>>>>> Hello Angel,
>>>>>
>>>>> since you now are a partner, you may have some information I don't get
>>>>> to see - but last time John Knowles was on IRC (~2 weeks ago) he indicated
>>>>> that the next ACS Community release would be a GA, so ACS 6.1 would have a
>>>>> Community equivalent.
>>>>>
>>>>> Regards, Axel
>>>>> On 26/11/2018 09:36, Angel Borroy wrote:
>>>>>
>>>>> Nice idea, Axel.
>>>>>
>>>>> I’m open to collaborate, despite my basic skills on Docker, k8s and
>>>>> many other relevant software involved in the “new” Alfresco.
>>>>>
>>>>> Now that Alfresco is ignoring deliberatively the Community (ACS 6.1
>>>>> will be published without having a Community equivalent), it’s time to act.
>>>>> Probably providing a deployment and developing environment for everyone is
>>>>> the right approach.
>>>>>
>>>>> I’m not able to organise this initiative, but let me know if I can
>>>>> help in any way.
>>>>>
>>>>> Best,
>>>>>
>>>>> Angel Borroy
>>>>> keen*soft *a *UST **Global *company
>>>>>
>>>>> email:   angel.borroy at keensoft.es
>>>>> web:     http://www.keensoft.es
>>>>> móvil:  +34 655 47 47 55 <//+34%20655%2047%2047%2055>
>>>>>
>>>>> CONFIDENCIALIDAD:
>>>>>
>>>>> La presente comunicación y, en su caso, los ficheros que lleve
>>>>> adjuntos, pertenecen exclusivamente a las personas a las que va dirigido y
>>>>> puede contener información confidencial. Si usted no es el destinatario de
>>>>> este mensaje (o la persona responsable de su entrega), considérese
>>>>> advertido de que lo ha recibido por error y que cualquier uso, difusión,
>>>>> reenvío o copia están prohibidos legalmente. Si ha recibido este mensaje
>>>>> por error, por favor notifíquelo al remitente y proceda a destruirlo
>>>>> inmediatamente.
>>>>>
>>>>> This message and the attached documents may contain
>>>>> privileged/confidential information and intended solely for the use of the
>>>>> individual to whom it is addressed. If you are not the intended recipient
>>>>> (or responsible for delivery of the message to such a person) be advised
>>>>> that you have received this message in error and that any use,
>>>>> dissemination, forwarding, printing or copying of this e-mail is strictly
>>>>> prohibited. If you have received this message in error please notify it to
>>>>> the sender and destroy it immediately.
>>>>>
>>>>> On 25 November 2018 at 22:57:06, Axel Faust (
>>>>> axel.faust.g at googlemail.com) wrote:
>>>>>
>>>>> Hello guys,
>>>>>
>>>>> TL;DR: Instead of revisiting our previous pre-bundled Community build
>>>>> idea, maybe more short-term / easier to realize value could be gained
>>>>> by
>>>>> collaborating on guides / documentation / examples regarding Docker
>>>>> deployments to help community members make the switch, AND/OR create a
>>>>> standard build setup (in lieu of an updated Alfresco SDK that may never
>>>>> arrive) incorporating the new deployment approach and achieving full
>>>>> compatibility with the most recent Alfresco releases.
>>>>>
>>>>> This mail is a revisit of a mail Daren sent in the middle of the thread
>>>>> for volunteers for the DevCon talk submission review ([1]). I don't
>>>>> know
>>>>> how many actually read the mail considering the misleading subject
>>>>> line.
>>>>> In it he asked if there is interest in revisiting our previous work
>>>>> around a pre-bundled Alfresco Community build (at the time using
>>>>> puppet)
>>>>> and maybe adapted / updating that to be in line with the current Docker
>>>>> / Kubernetes approach.
>>>>>
>>>>> In the last two months I have had the (dis-)pleasure to familiarize
>>>>> myself with the current state of the Alfresco Docker and Helm charts /
>>>>> Kubernetes work in preparation for a training at an Alfresco partner.
>>>>> Anyone following me on Twitter might have seen a rant or two. It
>>>>> certainly is not the most intuitive and not helped by the fact that
>>>>> Alfresco tends to break stuff shortly after providing an initially
>>>>> working state.
>>>>>
>>>>> Given how tricky it can be for community members / customers to switch
>>>>> over to the new approach without any prior experience, I would say that
>>>>> any guides / documentation that we as the Order could collaboratively
>>>>> put together would be much more valuable/needed at this point than
>>>>> simply revisiting the pre-bundled build. As both the Docker Compose
>>>>> scripts and Helm charts from Alfresco should only be considered as a
>>>>> starting point / reference rather than a ready-to-use setup, we might
>>>>> also want to look into providing improved variants for these (Helm may
>>>>> be a lower priority as it/Kubernetes likely is not relevant for most
>>>>> community users). I know that Angel has already done work in the area
>>>>> of
>>>>> an improved compose reference ([2]) - maybe that could be a starting
>>>>> point to flesh out / improve upon...
>>>>>
>>>>> In my training I have also integrated the new Docker images as a
>>>>> runtime
>>>>> environment for ("not-really-rapid") application development in my
>>>>> Maven
>>>>> setup, and also used it for the integration tests therein. In another,
>>>>> personal project I have started to use Docker containers even for my
>>>>> regular unit tests.
>>>>> Alfresco has still not released a SDK natively compatible with Alfresco
>>>>> 6, and given their long-term goal of moving customers/users off of
>>>>> in-process extensions may likely not provide an updated version anytime
>>>>> soon. I know most people in the Order have sufficient skill to adapt
>>>>> the
>>>>> current SDK to work with 6, but that is likely not true for most
>>>>> community users out there. So maybe we as the Order should look into
>>>>> providing a reference build setup of our own, using the new
>>>>> incorporating the new Docker approach as best as possible. And maybe
>>>>> such work will naturally lead us to general improvements / definition
>>>>> of
>>>>> a base image e.g. with regards to image size, code+test round-trip time
>>>>> etc...
>>>>>
>>>>> Regards
>>>>>
>>>>> Axel
>>>>>
>>>>>
>>>>> [1]
>>>>>
>>>>> http://lists.xtreamlab.net/pipermail/ootb-hive/2018-September/001028.html
>>>>>
>>>>> [2] https://github.com/keensoft/docker-alfresco
>>>>>
>>>>> _______________________________________________
>>>>> OOTB-hive mailing list
>>>>> OOTB-hive at lists.xtreamlab.net
>>>>> https://lists.xtreamlab.net/mailman/listinfo/ootb-hive
>>>>>
>>>>> _______________________________________________
>>>>> OOTB-hive mailing list
>>>>> OOTB-hive at lists.xtreamlab.net
>>>>> https://lists.xtreamlab.net/mailman/listinfo/ootb-hive
>>>>>
>>>> _______________________________________________
>>>> OOTB-hive mailing list
>>>> OOTB-hive at lists.xtreamlab.net
>>>> https://lists.xtreamlab.net/mailman/listinfo/ootb-hive
>>>>
>>>>
>>>> _______________________________________________
>>>> OOTB-hive mailing list
>>>> OOTB-hive at lists.xtreamlab.net
>>>> https://lists.xtreamlab.net/mailman/listinfo/ootb-hive
>>>>
>>>>
>>>> _______________________________________________
>>>> OOTB-hive mailing listOOTB-hive at lists.xtreamlab.nethttps://lists.xtreamlab.net/mailman/listinfo/ootb-hive
>>>>
>>>> _______________________________________________
>>>> OOTB-hive mailing list
>>>> OOTB-hive at lists.xtreamlab.net
>>>> https://lists.xtreamlab.net/mailman/listinfo/ootb-hive
>>>>
>>>> _______________________________________________
>>>> OOTB-hive mailing list
>>>> OOTB-hive at lists.xtreamlab.net
>>>> https://lists.xtreamlab.net/mailman/listinfo/ootb-hive
>>>>
>>> _______________________________________________
>>> OOTB-hive mailing list
>>> OOTB-hive at lists.xtreamlab.net
>>> https://lists.xtreamlab.net/mailman/listinfo/ootb-hive
>>>
>> _______________________________________________
> OOTB-hive mailing list
> OOTB-hive at lists.xtreamlab.net
> https://lists.xtreamlab.net/mailman/listinfo/ootb-hive
>
>
> _______________________________________________
> OOTB-hive mailing list
> OOTB-hive at lists.xtreamlab.net
> https://lists.xtreamlab.net/mailman/listinfo/ootb-hive
>

-- 
Andreas Steffan

Achter Billing 14
22399 Hamburg
Germany

skype: deas0815
M: +49 160 4694826
T: +49 40 23943542
F: +49 40 23943542

http://www.contentreich.de

Contentreich : Alfresco ECM, Clojure, Groovy und WordPress - aus Spaß und
für Geld
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.xtreamlab.net/pipermail/ootb-hive/attachments/20181207/73d3fa46/attachment-0001.html>