New post
AllAccepted answers
Luca Ravazzolo · Feb 5, 2016

Thanks for sharing more info Scott. Sorry I'm late on this. Travelling...

Deployment & management: This should be totally automated. There shouldn't be the need for a GUI (it slow things down). I've given my views on another thread/post you started on this exact subject. There is much to chew on these things and you might be under time pressure, however, it's an unavoidable point (automation) if we all want to be more competitive.

Your last paragraph (human error) highlights why we need to embrace more automation. So to answer the original post question: I'd put my complexity in automation :)

I understand what you're saying BTW; I wish you well with this workflow and the whole project.

All the best

Luca Ravazzolo · Feb 5, 2016

Scott:

it depends what you mean by a deployment tool. It opens up a whole new world of automation so you'll have to start to think about versioning your artifacts etc. It depends how you want to embrace your whole provisioning and deployment and management process.

My suggestion for infrastructure provisioning & deployment: Terraform; for management Ansible: agentless and easy to learn and use.

Of course, Puppet & Chef are strong valid alternatives but you'll have to take on board other considerations...

This subject cannot be done with one post and it really depends on how much automation you want to bring to this process. I would even suggest you consider containers. 

Once automation is seriously considered there is no turning back :-)

All the best

Luca Ravazzolo · Feb 2, 2016

Hi Scott,

Why do you want to use 1 single interface for multiple providers? Don't answer "why not" :-)

SPOF can be applied to anything. If that interface is critical and it goes down, it's like if your whole production, your all cluster and solution were down. Not good. It's just like in the microservices world. No difference and dangerous. Just like in the security world: it's not IF they'll attack you, it's just a matter of WHEN. Same here. Therefore, I need to ask you a second question: What is your service strategy for availability for the Production and this BS (assuming it's critical here)?

Personally, I would de-couple and leave the responsibility of files accuracy (name, timestamp, upload time-frame vs polling etc. or whatever else) to the customers. Do you have FIFO to respect?  Other considerations? 

Ensemble is ideal for having a centralized orchestrating BP handling all the incoming requests from the business services. Your BS implementation would be simpler as you implicitly would know what customer you're reading from. Be careful also of the number of files in the reading directory. OSs have limits and issues have been witnessed before (we don't know the numbers here but splitting in various DIRs alleviate the issue mentioned).

Anyway, it might just get down to personal preferences at the end or to one particular context variable we're not aware of here.

just some thoughts... there is no right or wrong in some of these architectures... just point of views, considerations, previous experiences and best practices...

HTH

Luca Ravazzolo · Feb 1, 2016

Hi Dmitry,

This year GS is going to be different. I'm sure there will be a session on Docker containers.

in the meantime, you might want to pick a Dockerfile example and the ccontainermain exe from here:

https://github.com/zrml/ccontainermain

If you have specific questions please feel free to open up a Docker specific thread under this Cloud group.

Thanks