Read, time-travel and append to Apache Iceberg tables, straight from R.
No Spark, no JVM, no SQL engine in the middle. 🧊
📦 Installation
# the CRAN release
install.packages("icebergr")
# or the development version, from r-universe
install.packages("icebergr", repos = c(
"https://pursuitofdatascience.r-universe.dev",
"https://cloud.r-project.org"
))On Windows and macOS that is a prebuilt binary. On Linux, R compiles it, so install Rust 1.88 or newer first:
🔍 Read
library(icebergr)
tbl <- icebergr_example_table() # a real Iceberg table, built on your machine
icebergr_collect(icebergr_scan(tbl, filter = amount > 997, select = c("id", "event", "amount")))
#> # A tibble: 3 × 3
#> id event amount
#> <int> <chr> <dbl>
#> 1 1498 refund 998
#> 2 1499 purchase 999
#> 3 1500 refund 1000The filter runs inside Iceberg, which skips whole files before reading a byte:
⏳ Time travel and ✍️ appends
first <- icebergr_snapshots(tbl)$snapshot_id[[1]]
nrow(icebergr_collect(icebergr_scan(tbl, snapshot_id = first)))
#> [1] 500
tbl <- icebergr_append(tbl, head(icebergr_collect(tbl), 2)) # a new snapshot, nothing rewritten
nrow(icebergr_collect(tbl))
#> [1] 1002🔌 Your own catalog
Put your token in ~/.Renviron as ICEBERGR_REST_TOKEN=... and restart R, then:
catalog <- icebergr_catalog("rest", uri = "https://your.catalog")
tbl <- icebergr_table(catalog, "db.events")AWS Glue, S3 and every other setting: catalog configuration.
🗺️ What works
| ✅ Works | 🚫 Not yet | |
|---|---|---|
| Read | filter and column pushdown · merge-on-read deletes · struct, list, map
|
🦀 limit pushdown · 🦀 filters on nested fields |
| Time travel | by snapshot id or time · the schema as of any snapshot | |
| Write | append · create a table or namespace · register a table | 🚧 partitioned appends · 🦀 deletes, MERGE, overwrite |
| Catalogs | REST · local memory · ⚙️ AWS Glue · ⚙️ S3 |
🦀 Hadoop (use memory) |
⚙️ opt-in at build time · 🦀 missing in iceberg-rust itself · 🚧 not in this release · icebergr_spec_support() lists everything for your build.
⚠️ Good to know
| 🔑 | Credentials come from environment variables, never arguments, and never travel over plain http://. |
| 🆔 | Snapshot ids are character: they are 64-bit integers, and a double holds 53 bits. |
| 🔢 | Install bit64 so long columns stay exact past 2^53. |
| 🧵 | Parallel work: parallel::makeCluster(), not mclapply(), and connect inside each worker. |
| 🐢 |
limit applies after the scan. To read less, filter. |