Going Serverless: AWS Lambda, and S3

Standard
Reading Time: < 1 minute

Setting up a local development environment for a serverless AWS system. I will be setting up a local development environment for development.

Things needed:
NVM
Node
AWS CLI tool Version 1
Serverless Framework

Install AWS CLI Version 1:

The guide to install can be found here.

Install NVM:

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.1/install.sh | bash

Install Node:

nvm install 10

nvm use 10

Install Serverless Framework:

npm install -g serverless

Create Project:

serverless create --template aws-nodejs --path projectName
cd projectName

Initialize NPM:

npm init

Node packages we’ll be using:

npm install aws-sdk --save-dev
npm install serverless-offline --save-dev
npm install serverless-http -save-dev
npm install serverless-s3-local --save-dev

AWS Profile (~/.aws/credentials):

[s3local]
aws_access_key_id = S3RVER
aws_secret_access_key = S3RVER

Using the profile:

export AWS_PROFILE=s3local

Testing and Using S3

Configure: (serverless.yml)

plugins:
  - serverless-s3-local
  - serverless-offline

custom:
  s3:
    port: 8081
    directory: ./tmp

resources:
  Resources:
    NewResources:
      Type: AWS::S3::Bucket
      Properties:
        BucketName: local-bucket

...

functions:
  s3hook:
    handler: handler.s3hook
    events:
      - s3: local-bucket

Edit handler.js:

...
module.exports.s3hook = async (event, context) => {
  console.log(JSON.stringify(event));
};

Start Serverless Offline: (try to use separate terminals)

sls offline start

Pushing up file to Local S3:

aws --endpoint http://localhost:8081 s3api put-object --bucket local-bucket --key handler.js --body ./handler.js

Retrieving a file from Local S3:

aws --endpoint http://localhost:8081 s3api get-object --bucket local-bucket --key handler.js ./tmp/handler.js

Results:

On your second terminal, the one you ran sls offline start you should have log outputs stating you have successful PUT and GET responses.

For the GET request, you should have a file in your tmp/ directory.

Upgrading Rails 5.2.2.1 to 6.0

Standard
Reading Time: < 1 minute

Update Gemfile

# update gem 'rails', '~> 5.2.2.1' to
gem 'rails', '~> 6.0.0'

# add
gem 'webpacker', '~> 4.0'

Grab updates & install webpack

bundle update

bin/rails webpacker:install

Run your test and fix any issues. I mostly got issues revolving around update_attributes and expanding where.not() queries.

I had to update a bunch of
@resource.update_attributes(resource_params)
to
@resource.update(resource_params).

I also had scopes that looked like:

scope :complete, -> { where.not(resume_id: nil, cover_letter_id: nil, company_id: nil) }

They had to be updated to:

scope :complete, -> { where.not(resume_id: nil).where.not(cover_letter_id: nil).where.not(company_id: nil) }

Also, I had an issue with one of my render partials.

Failure/Error: <%= render partial: :company_datum, collection: @company_data, locals: { offset: @company_data.offset }, cached: true %>

Had to update it to:

<%= render partial: 'company_datum', collection: @company_data, locals: { offset: @company_data.offset }, cached: true %>

Updating application.rb

# config/application.rb
# change config.load_defaults 5.2
config.load_defaults 6.0

# I removed
config.i18n.fallbacks = [I18n.default_locale]

If using rubocop: add this to your .rubocop.yml

AllCops:
  Exclude:
    - 'node_modules/**/*'
...

Fin.

Part 1: An API using Phoenix, Absinthe (GraphQL), Guardian, React, Apollo Client

Standard
Reading Time: 2 minutes

I love my Rails, but this is something I am really interested in.

It all started with a little bookkeeping app written using Phoenix as an API and React as a view layer. Has now turned into an extremely deep rabbit hole. After completing the first features of the app, I had the question “These states in these components are getting out of hand, I wonder if there’s something to help me?” This lead to Redux, one object to hold the state of the entire app. Then lead to the question, “Is there a better way to handle the data request to the API?” This lead to GraphQL which lead to Absinthe which made sense since I am using Phoenix as my API.

I think the best way to learn is to build something simple but not trivial. So let us begin with an e-commerce store that sales widgets.

Users >> Carts >> Widgets

Users can have carts to place widgets in them, etc.

I am going to break this down to 3 to 4 articles:
Part 1: Introduction to Phoenix, Absinthe, and Guardian – explaining the pieces.

Part 2: Implementation of the Phoenix/Absinthe API.
This is how you go about creating the API.

Part 3: Implementation of React/Redux with Apollo client.
We will create the frontend while still inside Phoenix, no additional steps Webpack will handle it.

Part 4: Additional nice things to have like: Elasticsearch, a Pub/Sub feature, etc. This one is a maybe.

Code can be found here.

https://github.com/mfung/widgets_r_us

What is Phoenix?

Phoenix is a web development framework similar in vein to Ruby on Rails. Phoenix is written in Elixir. Elixir is a functional, concurrent, and general-purpose programming language. Phoenix follows the MVC pattern and was developed to create highly performant and scalable web applications.

Laymen terms: You use it to build websites fast and efficiently.

What is GraphQL?

GraphQL was developed by Facebook in 2012 and released to the public in 2015. It is a data query and manipulation language for APIs.

Simply: An efficient and flexible way to get data, by eliminating unwanted data. I define what data I need from the endpoint.

What is Absinthe GraphQL?

Absinthe is “The GraphQL toolkit for Elixir.” Since Phoenix is written in Elixir and most APIs are web APIs. This just makes sense.

Guardian

If you are familiar with Devise in the Ruby world Guardian is similar. It is an authenication library for Elixir applications.

React

React is a JavaScript library used to build user interfaces for single-page applications and is optimal for fetching rapidly changing data.

Apollo GraphQL (Client)

Apollo comes in server and client. We will be dealing with the client. Apollo Client is a state management library for JavaScript GraphQL apps.

Next

docker-credential-pass Ubuntu setup for Bamboo CI/CD

Standard
Reading Time: < 1 minute

Ubuntu: 18.04
Docker: 18.09.5
docker-credential-pass: 0.6.2
pass: 1.7.1
gpg2: 2.2.4

Since my setup is for CI/CD I have a bamboo user to deploy my docker containers to Docker Hub and deploy to my production servers. Your use case may vary, but if you want your credentials to persist longer than a couple of hours before your system starts asking you to re-enter your passphrase this might work for you.

My gpg-agent.conf file looks like this:

This worked for me. Also note the default-cache-ttl value is in seconds, the values I used is 10 years in seconds.

Getting SimpleCov-summary (0.0.5) to work with SimpleCov (0.16.1) and Rails 5.2

Standard

Reading Time: < 1 minuteIf you run into issues with installing simplecov-summary using:

Here’s how to fix it:

Since Rails 5 removed silence_stream because it is not thread-safe. I found implementations of silence_stream but I got it to work by just removing it completely and I prefer to see these outputs at the end of my test run.

ADD NEW USER CENTOS, ADD TO SUDOER, ADD RSA KEYS

Standard

Reading Time: < 1 minute

 

Rsync Remote

Standard

Reading Time: < 1 minute

 

POSTGRESQL

Standard

Reading Time: < 1 minute

 

PALINDROME

Standard

Reading Time: < 1 minute

Ruby:

Python: