[OOTB-infra] misdirecting honeycomb instance (@heiko)

Martin Cosgrave martin at ocretail.com
Fri Jun 12 11:25:19 BST 2015 

No those VMs are not all necessary, what you can see there are multiple 
attempts to try to use ESXi for something useful.

Unfortunately:
* foreman cannot control ESXi VMs, only full vsphere ones
* jenkins can neither use ESXi slaves due to missing libraries in the 
free version
* showcase was an early attempt to have a showcase server managed by foreman
* qadci was supposed to be a 'quick and dirty CI' to try to use the 
resources for testing and CI, abandoned in favour of getting the release 
out in time. I don't think it has a running jenkins but whichever does 
have jenkins has multiple other services too

I have said from the start that ESXi was a bad choice and we should have 
chosen an open virtualisation platform. In use it has been a horrible 
experience trying to get anything done at all with it. The restrictions 
on the product which led to this horribly contorted network topology we 
have now have made it all but impossible to actually do any work on the 
infrastructure unless you use Windows, which I do not. And having a 
windows vm *inside* the ESXi does not actually help much unless you can 
get in to it easily, which I could not until I set up my own vpn and a 
guacamole server to redirect the windows vm to my web browser.

Daren mentioned that Honeycomb needs 'hardening', this is only due to 
the fact that the iptables script we used has a 'default open' policy 
for ports rather than 'default closed', which will be rectified as soon 
as I get the chance to work on it, but until that point it is handy for 
it to be behind the firewall. The machine in question is 'beehive'. 
Obviously 80 and 443 need to be open, and we also expose the various 
other ports like SMTP, FTP. Perhaps Daren can check out the full list of 
ports we expose, as I said I'm rather ill at the moment and I don't 
quite have the energy to track it down myself. (Nor do I have the energy 
for this conversation to be honest).

Before we do this though we should stop as a group as a whole and think, 
since there is nothing of use on this infrastructure at the moment (and 
I have wasted upwards of a hundred hours trying to get it to be useful) 
maybe we should reconsider tearing it down and replacing it with a KVM 
setup instead.

Martin

PS please don't divert the conversation into redmine, if you feel the 
need to raise issues the agreed way is to use the github issues page for 
the ootb-infra project.



On 12/06/15 10:07, Heiko Robert wrote:
> Martin,
>
> I ordered an additional IP. Unfortunately I have no idea where to 
> configure it since there is no documentation yet for the work you have 
> done so far. Could you please document the artefacts here:
> https://support.orderofthebee.org/projects/infra/wiki/
> you created and also all the logins (passwords in the keepass) / 
> installed software etc. so we are able to understand and manage this 
> stuff?
>
> What I can see is:
> * VM: beehive
> * VM: qadci with a running jenkins
> * VM: Foreman
> * VM: Jenkins CI
> * VM: Showcase
>
> additional there are the two VMs I created for you for testing:
> * VM: vm1
> * VM: vm2
>
> Are all these VMs really necessary? I understand the requirement for 
> one jenkins the foreman for your puppet stuff.
>
> Thanks
> Heiko
>
> Am 10.06.2015 um 23:43 schrieb Martin Cosgrave:
>> It's not a complaint, I am trying to ascertain how to get the honeycomb
>> build to work. The intention is to just get the box on the internet. I'm
>> not ignoring your questions but frankly I do not have the energy to get
>> involved in more config, I did a lot of work to get the honeycomb build
>> ready and in parallel I am trying to learn a new job environment, so I
>> don't really want to twist my mind round any complicated config at this
>> point.
>>
>> If you or anyone else in the infra team can help to get the published
>> artifact of the distro team visible on the infrastructure that would be
>> really helpful. If it involves modifying the honeycomb build to do so
>> well that's not happening for at least a month. In the meantime it's a
>> great shame not to be showing the output of the work that has been done.
>> Hence I mentioned hosting somewhere else. It seems simple, it's been
>> designed to run out of the box on a cloud VM like digital ocean or AWS,
>> if our infrastructure cannot provide a VM suitable for it to run on then
>> the obvious place is a cloud provider or similar.
>>
>>
>>
>> On 10/06/15 22:06, Heiko Robert wrote:
>>> Martin,
>>> you confuse me. What is/was the intension of your mails? Why are you
>>> not answering my questions? My intension is to help you getting your
>>> isse done but receive complaints.
>>> If you already decided to put your VM directly into the internet with
>>> a dedicated IP you may order another IP and bind it to your VM. I'm
>>> not the guy trying to convert you but you should be able to explain
>>> others why this is a good idea ...
>>>
>>> Heiko
>>>
>>> Am 10.06.2015 um 21:03 schrieb martin at ocretail.com:
>>>> OK I get it I think, the reason I'm getting a redirect error is 
>>>> because
>>>> of not returning that header? OK
>>>> The thing is, I'm not going to add that feature to the next point
>>>> release of honeycomb, perhaps it will be an option in 1.1.0 but right
>>>> now I need to run a production version honeycomb as it is. So I guess
>>>> we're going to have to host it elsewhere, it's a shame we didn't 
>>>> get any
>>>> extra IPs at the time we got the box, as I suggested at the time.
>>>>
>>>>  > On 09 June 2015 at 20:58 Heiko Robert
>>>> <heiko.orderofthebee.info at ecm4u.de> wrote:
>>>>  >
>>>>  >
>>>>  > Hi Martin,
>>>>  >
>>>>  > it's up to you:
>>>>  >
>>>>  > ** using reverse proxy chain **
>>>>  >
>>>>  > we/I could define an additional virtual host in our apache reverse
>>>> proxy
>>>>  > and make sure the required header attributes are set. No rewrite
>>>> rules
>>>>  > are set here to allow you max flexibility but SSL cert is
>>>> requrired on
>>>>  > the upfront apache proxy to handle SSL handshake. You should also
>>>> make
>>>>  > sure that in your virtual host the proxy headers are handled. What
>>>> are
>>>>  > you using - apache or nginx?
>>>>  > At the end the tomcat config needs to translate the header 
>>>> variables
>>>>  > back or you set hostname and port the hard way. Do you connect to
>>>> Tomcat
>>>>  > using http or ajp? In case of http you could define a connector
>>>> valve to
>>>>  > translate automatically:
>>>>  >
>>>>  > <!-- Connectors for reverse proxy (nginx) terminating SSL -->
>>>>  > <Connector port="8081" address="localhost" URIEncoding="UTF-8"
>>>>  > protocol="HTTP/1.1"
>>>>  > maxThreads="300" connectionTimeout="600000"
>>>>  > maxHttpHeaderSize="32768"
>>>>  > redirectPort="443" disableUploadTimeout="false"
>>>>  > proxyPort="443" scheme="https" secure="false" sslProtocol="TLS"
>>>>  > maxSavePostSize="-1"
>>>>  > />
>>>>  >
>>>>  >
>>>>  > <Valve className="org.apache.catalina.valves.RemoteIpValve"
>>>>  > remoteIpHeader="X-Forwarded-For"
>>>>  > remoteIpProxiesHeader="X-Forwarded-By"
>>>>  > protocolHeader="X-Forwarded-Proto"
>>>>  > />
>>>>  >
>>>>  > where
>>>>  > remoteIpHeader is the client IP header attribute and
>>>>  > remoteIpProxiesHeader is the proxy IPs header attribute
>>>>  >
>>>>  > There is absolutely no reason not to terminate HTTPS but it's 
>>>> just a
>>>>  > decision. Tomcat config has to be touched in any case.
>>>>  >
>>>>  >
>>>>  > ** using port forward **
>>>>  >
>>>>  > Port forwards are IP based and therfore don't support name 
>>>> dependant
>>>>  > resolution on the same IP. Define an unused port number of your
>>>> choice
>>>>  > and we could forward this port on the firewall to your reverse 
>>>> proxy
>>>>  > port to the VM.
>>>>  >
>>>>  > Make your decision and I can configure or show you how to do it.
>>>>  >
>>>>  > Heiko
>>>>  >
>>>>  > >> Do you have your own reverse proxy components (apache/nginx)?
>>>>  > > Yes that's exactly the issue
>>>>  >
>>>>  > > I just want to know if it is possible to have a virtual host 
>>>> proxy
>>>>  > > without any ssl termination or url rewriting, as our instance
>>>> already
>>>>  > > handles that.
>>>>  > _______________________________________________
>>>>  > OOTB-infra mailing list
>>>>  > OOTB-infra at xtreamlab.net
>>>>  > http://www.xtreamlab.net/mailman/listinfo/ootb-infra
>>

More information about the OOTB-infra mailing list