PeachCloud/meta
0
0
Fork 0
Code Issues 5 Pull Requests Projects Releases Wiki Activity
Files
master
meta/diary.md
Michael Williams c2eef0e2a6 retro diary
2018-06-11 10:51:50 +12:00

12 KiB
Raw Permalink Blame History

dev diary

2018-03-29

@dinosaur

  • setup ButtCloud.org and various linked services (GitHub, Docker Cloud, Twitter, Gmail)
  • started buttcloud/meta repo to store any meta information (including these dev diary entries)
  • started `buttcloud/butt-landing to be a simple public landing page for pub servers
    • working towards a multi-service pub using Docker Compose, i realized ssb-viewer and git-ssb-web were non-trivial to install in typical environments
    • the plan is to use this as a simple secondary service to get a multi-service pub working, then later can focus on the other secondary services

2018-03-30

@dinosaur

  • made Docker image for buttcloud/butt-landing
  • extracted minimal peer server and client code from scuttlebot into buttcloud/butt-peer
    • added custom logging plugin which uses pino, to try and have a consistent logging system across services
    • made two Docker images, one for butt-peer-server and one for butt-peer-client
      • i combined them so i could use the client code in the server healthcheck

2018-04-03

@dinosaur

  • started buttcloud/butt as docker-compose.yml of peer server, landing server, and nginx proxy
  • update butt-peer-server and butt-peer-client to not use node user, easier to start with default root user
    • why? i ran into an error with volume data permissions, would rather punt to later
    • UPDATE: changed this back, got it working with node user, needed to create volume in Dockerfile then mount external volume at same path
  • update butt-landing to auto re-connect to sbot, so it doesn't error when sbot is not yet up
    • why? this is the recommended way to have docker services depend on each other, docker-compose.yml v3 not longer supports depends_on: condition: healthy
  • setup Docker Hub automated builds to build tagged images based on git version tags
    • match tag name: /^v[0-9.]+$/, docker tag is same

2018-04-04

@dinosaur

  • got minimal buttcloud/butt working!
  • iron out some kinks...

2018-04-08

@dinosaur

  • switch to focus on Docker Swarm
  • change to use Traefik instead of Nginx Proxy

2018-04-09

@dinosaur

  • keep trying to get the Swarm setup to work, grr...!

2018-04-13

@dinosaur

  • take a break from Swarm for meow
  • first pass at scaffolding buttcloud/buttcloud-provider
    • have a working web server, browser app, and worker, but not yet complete to start feature development

2018-04-14

@dinosaur

  • more progress toward provider app scaffolding

buttcloud diary

2018-04-15 to 2018-04-17

@dinosaur

provider app has landed!

provider app landing

  • integrate redux-bundler
  • add emojis
  • found seamless background image to tile on landing page
  • start onboarding workflow
  • implement start step of onboarding
    • validate forms on client
    • validate service calls on server, show errors in form
    • show success or failure messages as snackbar
    • after form submission (which creates the user)
      • generate json web token that identifies user
      • send welcome email to next page of onboarding with token
        • emails are sent by queuing a delayed job to a worker (node-resque)
        • setup decent email templates with mjml
      • store user in local storage in case they refresh page before progressing
      • show help text on page
    • allow user to resend onboarding email

2018-04-18

@ahdinosaur

back to the infra side, during breakfast this morning i figured out why buttcloud/butt was failing!

i had a hunch that it had to do with the address that sbot was binding to. i configured host as example_butt-peer-server, since that's how the Docker service was to be identified within the Docker network. but still, the health checker inside the service couldn't find it. i changed this to 0.0.0.0 and it works!

did the same for the landing service. now the stack comes online, you can curl -H "Host: example.butt.nz" localhost and get the output from the landing page associated with example.butt.nz (proxied by traefik).

next i added a custom plugin to butt-peer-server that allows you to configure externalHost, in case it differs from host. this means we can bind to host (like 0.0.0.0) but advertise our public multiserver address as example.butt.nz (like for invites).

then, on a whim from @mischa, i went back to buttcloud-provider to swap redux-form for final-form, easy as.

made up some issues, want to step back to think about the next steps from here.

also made the ButtCloud logo!

ButtCloud logo

2018-04-19

@ahdinosaur

2018-04-26

@ahdinosaur

2018-04-27

@ahdinosaur

2018-04-30

@ahdinosaur

2018-05-01

@ahdinosaur

started docker-up: opinionated glue to manage our Docker swarm

2018-05-02

@ahdinosaur

continued with docker-up

  • ended up making a fun little continuable (cb => {}) based async flow control library in ./util/async.js, maybe will publish as flowstep
  • realized that the Docker API doesn't handle the docker stack * functionality, that's implemented in the Docker CLI
    • i learned that a "stack" is really a set of networks, volumes, and services each with a label "com.docker.stack.namespace" to reference the stack name
    • have to decide whether
      • a) to continue using the Docker API and implement that functionality ourselves
      • b) to move to using the Docker CLI
    • for now, will go with option a) !
      • reading the Docker CLI code, it's not scary or complex
      • this way we have more low-level control of the Docker Swarm
      • this way we can focus on exactly what we need for ButtCloud

2018-05-03

more docker-up, getting close to v1! 🎈

  • add executable cli
  • clean up the api
  • fractal stacks!
    • top-level config is a stack, with stacks all the way down
    • each stack has services, networks, volumes, AND NESTED STACKS
    • each stack may have a name to namespace associated services
  • pretty configurable logging

next up (notes to self):

  • better cli (take in resource type)
  • use explicit docker version in api requests
  • add "com.docker.stack.namespace" label to be legit docker stack

gotta work with @Mischa on another contract meow, then Art~Hack!

2018-05-04

@ahdinosaur

2018-05-07

@ahdinosaur

  • published docker-up/util/async.js as callstep, wrote up a splash of documentation 🚶‍♂️

2018-05-08

@ahdinosaur

  • docker-up: add basic integration and unit tests using ava, clean up log and config wrappers using composable callsteps: 29655b561b
    • cc @ike
  • butthub-provider: look into adding integration tests using codecept
  • not ButtCloud, but got sponsored by TickTack to improve ssb-pub: https://github.com/ahdinosaur/ssb-pub/pull/10

2018-05-09

@ahdinosaur

2018-05-10

@ahdinosaur

  • butthub-provider: battled some end-to-end testing dragons 🐉 : https://github.com/buttcloud/butthub-provider/pull/14
    • integrated the entire web app stack (api server, asset server, worker, and mailer) in the codecept process
    • at the end of the tests, the process was hanging, who was still running?
    • with some help from why-is-node-running and heaps of reading dependency internals, started the journey to find every remaining handle, gotta catch 'em all! 🐎

2018-05-11

@ahdinosaur

2018-05-15 - 2018-06-09

@ahdinosaur

  • am starting to finally understand some advanced fp (functional programming) concepts like monads
  • docker-up: investigate using sanctuary
  • docker-up: port code to use ramda, folktale, folktale-validations, and fluture
  • docker-up: re-architect how everything works
    • given next config, translate into docker api config
    • fetch all current docker api config
    • diff docker api current and next configs
    • show diff to sysadmin
    • if acceptable, translate the diff into docker api commands and run those
  • rename docker-up to gyne
  • rename ButtCloud to PeachCloud
  • get example swarm working across 3 virtual machines using docker-machine and gyne