Geecon 2017 Notes Day 1

19 Oct 2017

About

Geecon 2017 took place at cinema CineStar Cerny Most Prague, which was unsual location for conference at least for me personally. Lunch was little bit ackward to be eaten seated in the cinema.

Making It Count: Quality is Not Optional

Quality is cross cutting non functional requirement. Make explicit to agree with customer criticality. Take pride, ownership and responsibility for the project to encourage quality. Make people accountable. Establish naming. Spefication and docs are valuable. Early analyse requirements. Rise above implementation details and languages.

Reactive Programming

Reactive is non blocking event driven appsthat scale with small number of threads with backpressure as key ingredient that aims to ensure producers do not overhelm consumers.

Back pressure: consumers can prevent getting overwhelmed.

Main Abstractions

Publisher: Subscriber: Subscription: Processor<T, R>:

Reactive Core:

Spring Initializr

Spring Boot:

Demo: Stream Movies

Dependencies

Reactive Service

@SpringBootApplication
class Application {

  @Bean
  CommandLineRunner demoData (MovieRepo repo) {
    return args -> {
      repo.deleteAll().thenMany(  // insteadof .block()
          Flux
            .just("Silence of lambdas", "enter the void", "Blade Reactor")
            .map(title -> new Movie(title))
            .flatMap(m-> repo.save(m))
            subscribe(Sys.out::println)
           )
    }
  }


  @Bean
  RouterFunction<ServerResopnse> routerFunction
    RounterFunctions.route(Req\\Predicates.gET("movies")
      Mono<ServerResponse>handle ServerRequest req

  @Bean
  RouterFunction(movieHandler)
    route(gET("movies"), movieHandler::all
    route(GET...
    route(
}

@Component
MovieHandler
  movieServ

  all
    ServerResponse.ok()..body(movieServ.getAll(), Movie.class)
  byId
    ...
  events
    ...


@Service
class MovieService {
  MovieRepo repo

  MovieServ(repo)
    repo = repo

  Flux<Movie> getAllMovies{
    repo.findAll
  }

  Mono<Movie> getMovieById(id)
    repo.findById(id)

  Flux<MovieEvent> getEvents(id)
    Flux.generate(sink -> sink.next(new MovieEvent(id, new Date())))
      .delayElements(Duration.ofSeconds(1))
}

@RunWithSpingRunner
@SpringBootTest
MovieServiceTest
  MovieServ movieServ

  getEvenetsTake10
    id = serv.getAll.blockFirst.getId
    StepVerifier.withVirtualTime(() -> movieServ
      .getEvents(id)
      .take(10))
      .collectList()
      .thenAwait(10 hours)
      .assertNexxt(list -> assertTrue(list.size == 10)
      .verifyComplete

@RestController
@ReqeuestMapping "movies"
class MovieController
  movieServ

  @GetMapping
  Flux<Movie>all

  @GetM ${id}
  Mono<movie> byId(id)
    serv.getbyid id

  @GetM
    type = TEXT_EVENT_STREAM
    path = ${id}/events
  Flux<Event> events(id)
    serv.getevents id


@Document // For Spring Data
@Data // Lombok
@RequiredArgsConstructor
class Movie {
  @Id id
  @NonNull titile
}
}

@Data @NoArgsConst @AllArgsConstructor
class MovieEvent {
  movieId
  when
}

interface MovieRespository extends ReactiveCrudRepository

Reactive Client


@SBApp
Client
  @Bean
  WebClient client
   WebClient.create("localhost:8080/movies")

  @Bean
  CommandLineRunner demo(WebClient cllient)
    args -> client.get()
      .retrive
      .bodyToFlux(Movie.class)
      .filter(m -> m.getTitle.equalsIgnoroeCase("blade reactor"
      .flatMap(m -> client.get.uri("", m.getId).


MovieEvent
  id, when

Movie
  id, title

Refs

reactive-streams.org projectreacotr.io github.com/mkheck/flux-flix-intro github.com/joshlong/flux-flix-service

Java EE 8 Release

History

Started in 2014 Community surevey driven focus in 2016. Focus: Web, json, injection, servlet, jsf, json p, security, validation

Content

Modernize and simplify. Enhance cdi, web and security

json-p 1.1

Stamdard api to parse, generate, transform, query json Update to lateest standarts: pinter, patch, merge patch Add editing,…

Json pointer

string based syntax to id specific value

Json Patch

Json.createPatch
patch.apply(json)
Json.createPatchBuilder

Json Merge Patch

similar to patch. Those to be changed need to listed in the merge patch. Others will be lets untouched.

{a: va} + {b: vb} => {a: va, b:vb}
createMergeDiff

Json Collectors, Misc

iterate over json with collections skipArray, skipObject

Json Binding

similar to jaxb standard api for existing framework e.g. genson, gson

JsonbBuilder: entry point to JSON Binding api JsonB: fromJson, toJson

Default mapping: covers basic well known types Customizations: rich, ignorinng, null handling, instatntion, visibility, formatting Adapters n (De)Serializations can be implemented new JsonbConfig().withFormatting(true).withAdapters(new CarAdapter)

jax-rs

Serve-Sent Event

Browser receives auto updates from server via http. Not supported by IE.

Servlet

CDI

Bean Validation

Security API

Maintanance Release

Next?

Q & A

Deep dive and learn about your OS

Namespaces: Used by containers Linux capability: limiting accesses of executables Docker uses inherently linux. Docker on Win uses virtualization or something thinner to run Docker. Docker container is just a process on given linux.

Container isolation have side effect.

VT-X

Deved by Intel. Simulates separation using hardware cpu context switching. Other hardware like network cards are being switched or emulated.

Performance

Virtualization is not slow. People think that the virtualization is slow because because their experience with VirtualBox running full operating system. Issue there is that graphical card has to be emulated, which is slow.

Input and output is generally slow on virtualization, but CPU is very fast.

Virtualizaton Isolation

Virtualizattoin offers a lot more isolation. No ability to see other process, no shared limits. Much more security. Performance can be comparable if IO is controlled.

Alpine Linux

End Notes

Containers are not like VM. They have very limited isolation. Be aware. Understand the OS. Read on systemd. Coupon geeconPRG16. Be careful about OS upgrades. Database on containers is bad idea. Use linux everyday. Use Arch linux to learn by fixing it. ExHerbo distro very good to learn by much more breaking. Read Linux newsletter LWN. Read book OReilly Linux Kernel. Clever-Cloud.com

High Request System Dev

Case study about high req sys in Ad Tech.

Basics

Concepts

Call graphs

Requirements

Traffic fluctuates => auto scalable. Fault tolernace to nodes being down and connections lost. Safe to keep records, not losing revenue. Maintainable as expected to grow a lot. Observable to know what is happening to demo to both sides of the trades. Testability to avoid very costly bugs. Interoperable to be able to integrate with other various services without redeploys. Throughtput high to enable full trafic. Latency low to respond within 150ms with integrated sys.

Arch Evolution

AdEx -> LoadBalancer->Voluum DSP -> S3, Voluum Tracking Audience -> CDN -> S3

Added:

Bidders: match incoming reqs Bidding Helpers: special behaviour not to be in bidder Traffics: OpenRtb decode, Eureka: Allows discovery Processor: Voluum DBB Budget: records billing info Redis Campaign mgmt Cassandra

Initial Research and decisions

Json parsing: boon vs jackson databind vs jackson streaming.

Http engine: servlet vs netty vs jetty

Arch Pivots

Events Streaming used Amazon Kinesis Multi threading: thread per req vs single thread.

GraphQL API

Is a query language for apisand urntime for fulfilling those queries with your existing data. GrapQL provides a compelete and understandable desc of the dat ain your ap. Gives cliet ability to ask exactly what to know.

Filtering

This is only allows you to define your API. Filtering has to be defined and translated into language of the database.

Query {
  person( , args) {
    return Person.find({ where: args });
  }
}

Java Example

Spring Boot, GraphQL, Docker, Gradle

Schema contains objects and commands:

type Query {
  persons: [Person!]
  person(name: String!): Person
}

...

type Conference {
  id:ID!
  ...

Person is standards java pojo annotated with basic binding.

Query class implements GraphQ:QuerResolver and defines relations ships between queries and repositories?? Defines all same methods as defined in the schema file.

Example query:

query {
 conferences {
  name
  talks {
   title
   speakers {
     name
   }
  }
 }
}

JS Appolo

Also integrates with GQL. Similar implementation.

CQRS + Event Sources sys Live coding

CommandHandler takes command and return a result.

AddSessionCommmandHandler imiplements Handler
 handle(commnad)

Build Read model entity.

SessionFeedbackRoute
  getSessionFeedback(uuid)
    queryHandler.handles(new SessionFeedvackQuery(sessionId))
    session

Distrib sys

CAP Theorem

Different tradeoff discussed.

Example

CA key-val store

Two phase commit fails when single is down. But everybody is consistent. Every get is successful as long one is up.

PA key val

Commit succeeds as long one is up. Those that are down will for some time distribute outdated value.

CP key-val

System looks for majority agreement when getting. Setting values go through voted leaders.

Recommendations

Learn lingo and the basic. Do the research. Experiment with concepts.

Amazon dynamo Raft consensus algo github.com/bbossola/sysdist NoSQL Distilled

Design of Aeron System The Fastest Messaging System

Aeron was designed for very fast trading. Design. Evolution. We are in multicore era. Distributed computing. Connections can loose packets or arrive multiple times or in different order. New idea was to try to find alternative data structure. Similarly concurrency lead to immutable data structures. Concept of position was critical to the design. Flow and congestion always needs to controlled in messaging communication. Monitoring state we can slow down communication to avoid harmful states lead to speed improvements. Observability of the messaging system design improved debugging abilities.


Subscribe: Twitter , Facebook , RSS
Share on: Twitter , Facebook , Google+ , LinkedIn , Reddit .





Report any trackers to [email protected].