Georgina McFadyen
Elixir Umbrella Microservice or Majestic Monolith? Georgina McFadyen @gemcfadyen
Elixir Umbrella
@gemcfadyen
What is an Umbrella Project?
@gemcfadyen
Umbrella Application
@gemcfadyen
Create Umbrella
@gemcfadyen
Create Sub Applications
@gemcfadyen
Create Sub Applications
@gemcfadyen
Create Sub Applications
@gemcfadyen
sub app mix.exs
Root level config.exs
root_project/config/config.exs
@gemcfadyen
Run Tests
@gemcfadyen
Run Tests
@gemcfadyen
Dependencies
root_project/apps/sub_app_B/config/config.exs @gemcfadyen
Umbrella Summary •
Breaks down a complex system into parts
• •
Code hosted in one repository
•
Sub apps as internal dependencies
Sub apps can be developed independently
@gemcfadyen
Faros An Elixir Umbrella https://github.com/deploying-elixir/Faros
@gemcfadyen
Faros: An Elixir Umbrella Faros is a learning portal where blog posts can be saved and searches can be initiated @gemcfadyen
Landing Page
@gemcfadyen
Saving Posts
Searching
Faros Structure
@gemcfadyen
Communication
faros/apps/faros_frontend/mix.exs
@gemcfadyen
Communication
faros/apps/faros_frontend/mix.exs
@gemcfadyen
Why an Umbrella? Is it a sleek solution for Microservices or a Majestic Monolith?
@gemcfadyen
Lessons Learned
! ! @gemcfadyen
#1 Breaking Bad
@gemcfadyen
How do you define the boundary of each sub app?
@gemcfadyen
Core vs Web
@gemcfadyen
Per Responsibility
@gemcfadyen
Persistence
@gemcfadyen
One service per responsibility with one database service
@gemcfadyen
Umbrellas lend themselves to change
@gemcfadyen
#2 Elixir Versions
@gemcfadyen
@gemcfadyen
Requirements Requirements allow you to specify which versions of a given dependency you are willing to work against
https://hexdocs.pm/elixir/Version.html#module-requirements
@gemcfadyen
! @gemcfadyen
@gemcfadyen
:faros_frontend
elixir: “-> 1.2”
{:post_service, in_umbrella: true},
:post_service
elixir: “-> 1.4” @gemcfadyen
Strive For Consistency
@gemcfadyen
#3 Circular Dependencies
@gemcfadyen
Sub App 1
depends on
depends on
Sub App 2
Check if services are too granular - maybe they should be merged together !
#4 Transient Dependency Clashes
@gemcfadyen
@gemcfadyen
Wikipedia Dependencies
faros/apps/wikipedia/mix.exs
@gemcfadyen
Frontend Dependencies
faros/apps/faros_frontend/mix.exs
@gemcfadyen
Phoenix Dependencies
https://github.com/phoenixframework/phoenix/mix.exs
@gemcfadyen
@gemcfadyen
@gemcfadyen
Add Poison as an explicit dependency to Phoenix sub app
Use the override flag to ignore other versions
@gemcfadyen
#5 Namespace Clashes
@gemcfadyen
@gemcfadyen
wikipedia/search.ex
twitter/search.ex
@gemcfadyen
@gemcfadyen
! @gemcfadyen
@gemcfadyen
! @gemcfadyen
@gemcfadyen
💥 @gemcfadyen
Depending which module is loaded first depends on the outcome
@gemcfadyen
Namespace modules
@gemcfadyen
wikipedia/search.ex
twitter/search.ex
@gemcfadyen
#6 Test Data Cleanup
@gemcfadyen
Database Access 3 Umbrella sub applications interact with the database:
- db_service - post_service - faros_frontend @gemcfadyen
Database Access 3 Umbrella sub applications interact with the database:
- db_service - post_service - faros_frontend @gemcfadyen
Database Access 3 Umbrella sub applications interact with the database:
- db_service - post_service - faros_frontend @gemcfadyen
Ecto Sandbox to create Transactional Tests
@gemcfadyen
Configure the Sandbox
faros/apps/db_service/config/test.exs
@gemcfadyen
Get Connection
@gemcfadyen
Ownership
@gemcfadyen
Shared Mode
@gemcfadyen
Shared Mode
@gemcfadyen
#7 Testing Boundaries
@gemcfadyen
faros_frontend
wikipedia
@gemcfadyen
Substitute External Systems
@gemcfadyen
search_controller_test.exs
faros/apps/faros_frontend/test/controllers/search_controller_test.exs
search_controller.ex
faros/apps/faros_frontend/web/controllers/search_controller.ex
search_controller.ex
@gemcfadyen
Wrap the real search functionality
@gemcfadyen
@gemcfadyen
Provide a fake search @gemcfadyen
@gemcfadyen
MIX_ENV=test
MIX_ENV=dev
@gemcfadyen
fake_search.ex
third_party_search.ex
faros_frontend/config/test.exs
faros_frontend/config/dev.exs
@gemcfadyen
search_controller_test.exs
search_controller.ex
search_controller_test.exs
fake_search.ex
search_controller_test.exs
fake_search.ex
Test Doubles Be aware of the boundaries you cross
Switch real/fake implementations in/out
Test away from external systems
Deterministic tests
@gemcfadyen
#8 Documentation
@gemcfadyen
Limited Documentation Dig deep for answers
Read through issue lists
Scour blog posts for workarounds
Tooling documentation is playing catch up @gemcfadyen
Deployment
@gemcfadyen
Distillery Generate release config:
@gemcfadyen
Distillery Bundles all the sub applications into one single release:
faros/rel/config.exs @gemcfadyen
Distillery If you prefer to release each sub app individually:
@gemcfadyen
Any various combinations in between:
faros/rel/config.exs @gemcfadyen
Build & Start Release
@gemcfadyen
Start Release
@gemcfadyen
Build & Start Release Build release using distillery task:
Build specific release:
@gemcfadyen
Assets
@gemcfadyen
One release of all sub apps
or
Group sub apps and release separately
@gemcfadyen
Distribution
@gemcfadyen
Communication The Elixir repl is an individual Erlang runtime:
@gemcfadyen
Distributed Mode
@gemcfadyen
Distributed Mode
@gemcfadyen
Distributed Mode
@gemcfadyen
Cluster
@gemcfadyen
Connecting Nodes Node connection is transitive
@gemcfadyen
Communication A Node is a system running the Erlang VM with a given name
@gemcfadyen
Distribution
@gemcfadyen
Distribution wikipedia
@gemcfadyen
twitter
Distribution [email protected]
:
@gemcfadyen
[email protected]
:
Communication Services need to communicate using a protocol
@gemcfadyen
rpc A remote procedure call is a method to call a function on a remote node and collect the answer.
@gemcfadyen
Distributed Faros
@gemcfadyen
Cluster Configuration
@gemcfadyen
Cluster Configuration
faros/apps/faros_frontend/config/dev.exs
@gemcfadyen
Connect to Node
@gemcfadyen
Distribute to the Node
Cluster Configuration
@gemcfadyen
Remotely Distributed Faros
@gemcfadyen
Build Multiple Releases
@gemcfadyen
Deploy wikipedia release
@gemcfadyen
Configure the address of the node that wikipedia is deployed to
@gemcfadyen
Deploy faros release configured with that child node
@gemcfadyen
Elixir Umbrella Microservice or Majestic Monolith?
@gemcfadyen
Development Monlith: Single repository
Microservices: Separate repositories
Umbrella: Single repository
@gemcfadyen
Communication Monlith: Direct access
Microservices: Defined Protocol
Umbrella: Direct access to dependant sub applications
@gemcfadyen
Languages Monlith: Single language
Microservices: Can use different languages
Umbrella: Single language
@gemcfadyen
Releases Monlith: Shipped as one product
Microservices: Separate pieces shipped independently Umbrella: As you define
@gemcfadyen
Summary Less coordination of pull requests Easy to abstract libraries Flexibility in deployment
@gemcfadyen
Majestic Microservices
@gemcfadyen
Thank-you Elixir.LDN!
@gemcfadyen
Elixir Umbrella
@gemcfadyen
What is an Umbrella Project?
@gemcfadyen
Umbrella Application
@gemcfadyen
Create Umbrella
@gemcfadyen
Create Sub Applications
@gemcfadyen
Create Sub Applications
@gemcfadyen
Create Sub Applications
@gemcfadyen
sub app mix.exs
Root level config.exs
root_project/config/config.exs
@gemcfadyen
Run Tests
@gemcfadyen
Run Tests
@gemcfadyen
Dependencies
root_project/apps/sub_app_B/config/config.exs @gemcfadyen
Umbrella Summary •
Breaks down a complex system into parts
• •
Code hosted in one repository
•
Sub apps as internal dependencies
Sub apps can be developed independently
@gemcfadyen
Faros An Elixir Umbrella https://github.com/deploying-elixir/Faros
@gemcfadyen
Faros: An Elixir Umbrella Faros is a learning portal where blog posts can be saved and searches can be initiated @gemcfadyen
Landing Page
@gemcfadyen
Saving Posts
Searching
Faros Structure
@gemcfadyen
Communication
faros/apps/faros_frontend/mix.exs
@gemcfadyen
Communication
faros/apps/faros_frontend/mix.exs
@gemcfadyen
Why an Umbrella? Is it a sleek solution for Microservices or a Majestic Monolith?
@gemcfadyen
Lessons Learned
! ! @gemcfadyen
#1 Breaking Bad
@gemcfadyen
How do you define the boundary of each sub app?
@gemcfadyen
Core vs Web
@gemcfadyen
Per Responsibility
@gemcfadyen
Persistence
@gemcfadyen
One service per responsibility with one database service
@gemcfadyen
Umbrellas lend themselves to change
@gemcfadyen
#2 Elixir Versions
@gemcfadyen
@gemcfadyen
Requirements Requirements allow you to specify which versions of a given dependency you are willing to work against
https://hexdocs.pm/elixir/Version.html#module-requirements
@gemcfadyen
! @gemcfadyen
@gemcfadyen
:faros_frontend
elixir: “-> 1.2”
{:post_service, in_umbrella: true},
:post_service
elixir: “-> 1.4” @gemcfadyen
Strive For Consistency
@gemcfadyen
#3 Circular Dependencies
@gemcfadyen
Sub App 1
depends on
depends on
Sub App 2
Check if services are too granular - maybe they should be merged together !
#4 Transient Dependency Clashes
@gemcfadyen
@gemcfadyen
Wikipedia Dependencies
faros/apps/wikipedia/mix.exs
@gemcfadyen
Frontend Dependencies
faros/apps/faros_frontend/mix.exs
@gemcfadyen
Phoenix Dependencies
https://github.com/phoenixframework/phoenix/mix.exs
@gemcfadyen
@gemcfadyen
@gemcfadyen
Add Poison as an explicit dependency to Phoenix sub app
Use the override flag to ignore other versions
@gemcfadyen
#5 Namespace Clashes
@gemcfadyen
@gemcfadyen
wikipedia/search.ex
twitter/search.ex
@gemcfadyen
@gemcfadyen
! @gemcfadyen
@gemcfadyen
! @gemcfadyen
@gemcfadyen
💥 @gemcfadyen
Depending which module is loaded first depends on the outcome
@gemcfadyen
Namespace modules
@gemcfadyen
wikipedia/search.ex
twitter/search.ex
@gemcfadyen
#6 Test Data Cleanup
@gemcfadyen
Database Access 3 Umbrella sub applications interact with the database:
- db_service - post_service - faros_frontend @gemcfadyen
Database Access 3 Umbrella sub applications interact with the database:
- db_service - post_service - faros_frontend @gemcfadyen
Database Access 3 Umbrella sub applications interact with the database:
- db_service - post_service - faros_frontend @gemcfadyen
Ecto Sandbox to create Transactional Tests
@gemcfadyen
Configure the Sandbox
faros/apps/db_service/config/test.exs
@gemcfadyen
Get Connection
@gemcfadyen
Ownership
@gemcfadyen
Shared Mode
@gemcfadyen
Shared Mode
@gemcfadyen
#7 Testing Boundaries
@gemcfadyen
faros_frontend
wikipedia
@gemcfadyen
Substitute External Systems
@gemcfadyen
search_controller_test.exs
faros/apps/faros_frontend/test/controllers/search_controller_test.exs
search_controller.ex
faros/apps/faros_frontend/web/controllers/search_controller.ex
search_controller.ex
@gemcfadyen
Wrap the real search functionality
@gemcfadyen
@gemcfadyen
Provide a fake search @gemcfadyen
@gemcfadyen
MIX_ENV=test
MIX_ENV=dev
@gemcfadyen
fake_search.ex
third_party_search.ex
faros_frontend/config/test.exs
faros_frontend/config/dev.exs
@gemcfadyen
search_controller_test.exs
search_controller.ex
search_controller_test.exs
fake_search.ex
search_controller_test.exs
fake_search.ex
Test Doubles Be aware of the boundaries you cross
Switch real/fake implementations in/out
Test away from external systems
Deterministic tests
@gemcfadyen
#8 Documentation
@gemcfadyen
Limited Documentation Dig deep for answers
Read through issue lists
Scour blog posts for workarounds
Tooling documentation is playing catch up @gemcfadyen
Deployment
@gemcfadyen
Distillery Generate release config:
@gemcfadyen
Distillery Bundles all the sub applications into one single release:
faros/rel/config.exs @gemcfadyen
Distillery If you prefer to release each sub app individually:
@gemcfadyen
Any various combinations in between:
faros/rel/config.exs @gemcfadyen
Build & Start Release
@gemcfadyen
Start Release
@gemcfadyen
Build & Start Release Build release using distillery task:
Build specific release:
@gemcfadyen
Assets
@gemcfadyen
One release of all sub apps
or
Group sub apps and release separately
@gemcfadyen
Distribution
@gemcfadyen
Communication The Elixir repl is an individual Erlang runtime:
@gemcfadyen
Distributed Mode
@gemcfadyen
Distributed Mode
@gemcfadyen
Distributed Mode
@gemcfadyen
Cluster
@gemcfadyen
Connecting Nodes Node connection is transitive
@gemcfadyen
Communication A Node is a system running the Erlang VM with a given name
@gemcfadyen
Distribution
@gemcfadyen
Distribution wikipedia
@gemcfadyen
Distribution [email protected]
:
@gemcfadyen
[email protected]
:
Communication Services need to communicate using a protocol
@gemcfadyen
rpc A remote procedure call is a method to call a function on a remote node and collect the answer.
@gemcfadyen
Distributed Faros
@gemcfadyen
Cluster Configuration
@gemcfadyen
Cluster Configuration
faros/apps/faros_frontend/config/dev.exs
@gemcfadyen
Connect to Node
@gemcfadyen
Distribute to the Node
Cluster Configuration
@gemcfadyen
Remotely Distributed Faros
@gemcfadyen
Build Multiple Releases
@gemcfadyen
Deploy wikipedia release
@gemcfadyen
Configure the address of the node that wikipedia is deployed to
@gemcfadyen
Deploy faros release configured with that child node
@gemcfadyen
Elixir Umbrella Microservice or Majestic Monolith?
@gemcfadyen
Development Monlith: Single repository
Microservices: Separate repositories
Umbrella: Single repository
@gemcfadyen
Communication Monlith: Direct access
Microservices: Defined Protocol
Umbrella: Direct access to dependant sub applications
@gemcfadyen
Languages Monlith: Single language
Microservices: Can use different languages
Umbrella: Single language
@gemcfadyen
Releases Monlith: Shipped as one product
Microservices: Separate pieces shipped independently Umbrella: As you define
@gemcfadyen
Summary Less coordination of pull requests Easy to abstract libraries Flexibility in deployment
@gemcfadyen
Majestic Microservices
@gemcfadyen
Thank-you Elixir.LDN!
@gemcfadyen
