Elasticsearch vs Time-Series DB: Key Differences Explained

Scenario context

Elasticsearch is built for search — full-text log queries, fuzzy matching, and relevance ranking via an inverted index. InfluxDB and Prometheus are built for metrics — numeric time series with aggressive compression. Picking the wrong one wastes 10× the storage or makes queries 100× slower.

"failed" → [doc_3, doc_7, doc_12] "database" → [doc_3, doc_9] "connection" → [doc_7, doc_12, doc_20]

**Observable outcome:** Strengths: - Full-text search with stemming, fuzzy matching, synonyms - Relevance ranking (BM25) - Aggregation pipelines (histograms, top-N, date histograms) - Schema flexibility (dynamic mappings) Weaknesses: - High st…

## Stage 2 — Time-Series DBs: Delta Encoding and Columnar Compression

- TSDBs (InfluxDB, Prometheus, TimescaleDB, VictoriaMetrics) are optimized for the fact that metric values change slowly.
- Delta encoding example: Storing deltas instead of absolute values reduces the integer size dramatically.
- A 64-bit value becomes a 1-bit delta.
- With additional compression (Gorilla encoding, Snappy), modern TSDBs achieve 1–2 bytes per data point versus Elasticsearch's 50–100 bytes per log document.

*Applies [**sorting-and-searching-algorithms**](/concept/sorting-and-searching-algorithms) to the decision above.*

Raw: 100, 101, 102, 103 Encoded: 100, +1, +1, +1

```mermaid
flowchart LR
    Sensor[Sensor 87.3 87.4 87.5] --> Delta[Delta Encoding 87.3 +0.1 +0.1]
    Delta --> Compress[Gorilla XOR Compression]
    Compress --> TSDB[(TSDB Block 1-2 bytes/point)]

Observable outcome: With additional compression (Gorilla encoding, Snappy), modern TSDBs achieve 1–2 bytes per data point versus Elasticsearch's 50–100 bytes per log document.

Stage 3 — Data Flow: From Ingestion to Query

• In real production systems these two database types live side by side.
• The ELK stack (Elasticsearch + Logstash + Kibana) handles unstructured log events.
• The Prometheus + Grafana stack handles numeric metrics.
• Each path is optimized for its data shape: The log pipeline parses unstructured text, indexes every field, and stores full events so engineers can run ad-hoc text searches during an incident.

flowchart LR
    App[Application] -->|log events| LS[Logstash / Filebeat]
    LS --> ES[(Elasticsearch)]
    ES --> Kibana[Kibana Dashboard]

    App -->|numeric metrics| Prom[Prometheus Scraper]
    Prom --> InfluxDB[(InfluxDB / TSDB)]
    InfluxDB --> Grafana[Grafana Dashboard]

flowchart TD
    Start[What type of data?] --> Text{Is the data text or events?}
    Text -->|Yes: log lines, audit trails| ES[Use Elasticsearch Inverted index search]
    Text -->|No: numeric measurements| Num{Regular interval samples?}
    Num -->|Yes: every 15s, counters| TSDB[Use Prometheus / InfluxDB Delta encoding + rollups]
    Num -->|No: irregular events| Mixed{Need full-text search too?}
    Mixed -->|Yes| ES
    Mixed -->|No| TSDB
    ES --> Kibana[Visualize with Kibana KQL queries]
    TSDB --> Grafana[Visualize with Grafana PromQL queries]

Observable outcome: Each path is optimized for its data shape: The log pipeline parses unstructured text, indexes every field, and stores full events so engineers can run ad-hoc text searches during an incident.

Stage 4 — Real-World Application: Which One to Use and When

• In practice: Production observability stacks often use both.
• The ELK stack (Elasticsearch + Logstash + Kibana) handles logs; Prometheus + Grafana handles metrics.

Observable outcome: The ELK stack (Elasticsearch + Logstash + Kibana) handles logs; Prometheus + Grafana handles metrics.

Stage 5 — Practical: Setting Up Your Observability Stack

• A beginner-friendly starting point is the ELK + Prometheus/Grafana split .
• Here is the decision process for each type of signal: Step 1 — Classify your signal.
• Ask: Is this a sentence or a number?
• If your application writes structured log lines ( ERROR user=42 action=checkout latency ms=1200 ), those go to Elasticsearch.

Observable outcome: If your application writes structured log lines ( ERROR user=42 action=checkout latency ms=1200 ), those go to Elasticsearch.

Stage 6 — Spring Data Elasticsearch & InfluxDB Java Client: Querying Logs and Metrics from Java

• Spring Data Elasticsearch brings the familiar Spring Data @Repository abstraction to Elasticsearch — replacing hand-crafted JSON queries with method-name conventions and @Query annotations.
• InfluxDB's Java client ( influxdb-client-java ) lets you write timestamped Point objects and run Flux aggregation queries without leaving Java — mirroring the two-pipeline architecture (ES for logs, TSDB for metrics) de…
• For a full deep-dive on Spring Data Elasticsearch, InfluxDB, and TimescaleDB Java integration, a dedicated follow-up post is planned.

// ─── Spring Data Elasticsearch: search log events ─────────────────────────────
// build.gradle: org.springframework.boot:spring-boot-starter-data-elasticsearch

@Document(indexName = "nginx-logs")
public record NginxLog(
    @Id String id,
    int status,
    String path,
    Instant timestamp
) {}

// Method name generates the ES query automatically — no JSON needed.
// Under the hood: Elasticsearch executes an inverted-index term-range query
// across billions of log documents in milliseconds.
public interface NginxLogRepository
        extends ElasticsearchRepository<NginxLog, String> {
    List<NginxLog> findByStatusGreaterThanEqualAndTimestampBetween(
            int status, Instant from, Instant to);
    // ↑ translates to: "find all HTTP 5xx errors in the last hour"
    //   Full-text inverted-index query — this is where Elasticsearch shines.
}

// ─── InfluxDB Java client: write a CPU metric and query averages ───────────────
// build.gradle: com.influxdb:influxdb-client-java:7.1.0

import com.influxdb.client.*;
import com.influxdb.client.domain.WritePrecision;
import com.influxdb.client.write.Point;

InfluxDBClient client = InfluxDBClientFactory.create(
    "http://localhost:8086", token, org, bucket);

// Write a data point — delta-encoded numeric stream (maps to ⚙️ section)
client.getWriteApiBlocking().writePoint(
    Point.measurement("cpu_usage")
         .addTag("host", "web-01")
         .addField("percent", 87.3)
         .time(Instant.now(), WritePrecision.MS)
);

// Query: average CPU over the last hour using Flux
String flux = """
    from(bucket: "metrics")
      |> range(start: -1h)
      |> filter(fn: (r) => r._measurement == "cpu_usage")
      |> mean()
    """;

client.getQueryApi()
      .query(flux)
      .forEach(table -> table.getRecords()
               .forEach(r -> System.out.println("Avg CPU: " + r.getValue())));

client.close();

Observable outcome: For a full deep-dive on Spring Data Elasticsearch, InfluxDB, and TimescaleDB Java integration, a dedicated follow-up post is planned.

End-to-end outcome

A log is a sentence: 2024-01-15 ERROR: failed to connect to database host=db1 .

Alternatives considered

Logs vs Metrics: Two Different Storage Problems

A log is a sentence: 2024-01-15 ERROR: failed to connect to database host=db1 .

The Basics: Search Engines vs Time-Series Databases

Before diving into how each system is built, it helps to understand the fundamental contract each one offers.

Failure and degraded modes

Trade-offs & Failure Modes: Trade-offs, Failure Modes & Decision Guide: Cardinality and TSDB Limits

• Detection: The biggest operational risk in TSDBs is high-cardinality labels .
• User or operator impact: The biggest operational risk in TSDBs is high-cardinality labels .