transport.asciidoc

Transport

The elasticsearch-transport library provides a low-level Ruby client for connecting to an {es} cluster.

It handles connecting to multiple nodes in the cluster, rotating across connections, logging and tracing requests and responses, maintaining failed connections, discovering nodes in the cluster, and provides an abstraction for data serialization and transport.

It does not handle calling the {es} API.

For optimal performance, use a HTTP library which supports persistent ("keep-alive") connections, such as patron or Typhoeus. Require the library (require 'patron') in your code, and it will be automatically used.

Installation

Install the package from Rubygems:

gem install elasticsearch-transport

To use an unreleased version, either add it to your Gemfile for Bundler:

gem 'elasticsearch-transport', git: 'git://github.com/elasticsearch/elasticsearch-ruby.git'

or install it from a source code checkout:

git clone https://github.com/elasticsearch/elasticsearch-ruby.git
cd elasticsearch-ruby/elasticsearch-transport
bundle install
rake install

Example usage

In the simplest form, connect to {es} running on http://localhost:9200 without any configuration:

require 'elasticsearch/transport'

client = Elasticsearch::Client.new
response = client.perform_request('GET', '_cluster/health')
# => #<Elasticsearch::Transport::Transport::Response:0x007fc5d506ce38 @status=200, @body={ ... } >

Full documentation is available at http://rubydoc.info/gems/elasticsearch-transport.

Transport implementations

By default, the client uses the Faraday HTTP library as a transport implementation.

It auto-detects and uses an adapter for Faraday based on gems loaded in your code, preferring HTTP clients with support for persistent connections.

To use the Patron HTTP, for example, require it:

require 'patron'

Then, create a new client, and the Patron gem will be used as the "driver":

client = Elasticsearch::Client.new

client.transport.connections.first.connection.builder.adapter
# => Faraday::Adapter::Patron

10.times do
  client.nodes.stats(metric: 'http')['nodes'].values.each do |n|
    puts "#{n['name']} : #{n['http']['total_opened']}"
  end
end

# => Stiletoo : 24
# => Stiletoo : 24
# => Stiletoo : 24
# => ...

To use a specific adapter for Faraday, pass it as the adapter argument:

client = Elasticsearch::Client.new adapter: :net_http_persistent

client.transport.connections.first.connection.builder.handlers
# => [Faraday::Adapter::NetHttpPersistent]

To pass options to the Faraday::Connection constructor, use the transport_options key:

client = Elasticsearch::Client.new transport_options: {
  request: { open_timeout: 1 },
  headers: { user_agent:   'MyApp' },
  params:  { :format => 'yaml' },
  ssl:     { verify: false }
}

To configure the Faraday instance directly, use a block:

require 'patron'

client = Elasticsearch::Client.new(host: 'localhost', port: '9200') do |f|
  f.response :logger
  f.adapter  :patron
end

You can use any standard Faraday middleware and plugins in the configuration block.

You can also initialize the transport class yourself, and pass it to the client constructor as the transport argument:

require 'patron'

transport_configuration = lambda do |f|
  f.response :logger
  f.adapter  :patron
end

transport = Elasticsearch::Transport::Transport::HTTP::Faraday.new \
  hosts: [ { host: 'localhost', port: '9200' } ],
  &transport_configuration

# Pass the transport to the client
#
client = Elasticsearch::Client.new transport: transport

Instead of passing the transport to the constructor, you can inject it at run time:

# Set up the transport
#
faraday_configuration = lambda do |f|
  f.instance_variable_set :@ssl, { verify: false }
  f.adapter :excon
end

faraday_client = Elasticsearch::Transport::Transport::HTTP::Faraday.new \
  hosts: [ { host: 'my-protected-host',
             port: '443',
             user: 'USERNAME',
             password: 'PASSWORD',
             scheme: 'https'
          }],
  &faraday_configuration

# Create a default client
#
client = Elasticsearch::Client.new

# Inject the transport to the client
#
client.transport = faraday_client

You can also use a bundled Curb based transport implementation:

require 'curb'
require 'elasticsearch/transport/transport/http/curb'

client = Elasticsearch::Client.new transport_class: Elasticsearch::Transport::Transport::HTTP::Curb

client.transport.connections.first.connection
# => #<Curl::Easy http://localhost:9200/>

It’s possible to customize the Curb instance by passing a block to the constructor as well (in this case, as an inline block):

transport = Elasticsearch::Transport::Transport::HTTP::Curb.new \
  hosts: [ { host: 'localhost', port: '9200' } ],
  & lambda { |c| c.verbose = true }

client = Elasticsearch::Client.new transport: transport

You can write your own transport implementation by including the {Elasticsearch::Transport::Transport::Base} module, implementing the required contract, and passing it to the client as the transport_class parameter – or by injecting it directly.

Transport architecture

• Elasticsearch::Transport::Client is composed of Elasticsearch::Transport::Transport.

• Elasticsearch::Transport::Transport is composed of Elasticsearch::Transport::Transport::Connections, and an instance of logger, tracer, serializer and sniffer.

• Logger and tracer can be any object conforming to Ruby logging interface, for example, an instance of Logger, log4r, logging, and so on.

• The Elasticsearch::Transport::Transport::Serializer::Base implementations handle converting data for {es} (for example, to JSON). You can implement your own serializer.

• Elasticsearch::Transport::Transport::Sniffer allows to discover nodes in the cluster and use them as connections.

• Elasticsearch::Transport::Transport::Connections::Collection is composed of Elasticsearch::Transport::Transport::Connections::Connection instances and a selector instance.

• Elasticsearch::Transport::Transport::Connections::Connection contains the connection attributes such as hostname and port, as well as the concrete persistent "session" connected to a specific node.

• The Elasticsearch::Transport::Transport::Connections::Selector::Base implementations allow to choose connections from the pool, for example, in a round-robin or random fashion. You can implement your own selector strategy.