Selecting and filtering¶
Columns¶
Select keeps the columns it names, in its order. Col names columns,
ColRegex matches them by a pattern, and ColDType by their type, so the same query
adapts when the schema grows:
df, err := sales().
Select(
ursus.Col("order_id"),
ursus.ColRegex("^(qty|price)$"),
ursus.ColDType(ursus.String),
).
Head(3).
Collect(context.Background())
if err != nil {
log.Fatal(err)
}
fmt.Println(df)
shape: (3, 5)
┌──────────┬───────┬─────────┬─────────┬─────────┐
│ order_id │ qty │ price │ region │ product │
│ Int64 │ Int64 │ Float64 │ String │ String │
├──────────┼───────┼─────────┼─────────┼─────────┤
│ 1 │ 10 │ 1.5 │ "north" │ "apple" │
│ 2 │ 4 │ 1.5 │ "south" │ "apple" │
│ 3 │ 7 │ 2.25 │ "north" │ "pear" │
└──────────┴───────┴─────────┴─────────┴─────────┘
Exclude is every column but the named ones:
df, err := sales().
Select(ursus.Exclude("order_id", "date")).
Head(3).
Collect(context.Background())
if err != nil {
log.Fatal(err)
}
fmt.Println(df)
shape: (3, 4)
┌─────────┬─────────┬───────┬─────────┐
│ region │ product │ qty │ price │
│ String │ String │ Int64 │ Float64 │
├─────────┼─────────┼───────┼─────────┤
│ "north" │ "apple" │ 10 │ 1.5 │
│ "south" │ "apple" │ 4 │ 1.5 │
│ "north" │ "pear" │ 7 │ 2.25 │
└─────────┴─────────┴───────┴─────────┘
New in 0.6.7: package selector picks columns by what they are, as Polars'
selectors do: Numeric, Integer, Float, Temporal, String and the other
types; StartsWith, EndsWith, Contains and Matches by name; ByName,
ByIndex, First and Last; and their Union, Intersect, Difference,
SymmetricDifference and Complement, Polars' |, &, -, ^ and ~, named
so that a selector's complement is never taken for an expression's negation, and
Exclude. A selector's Expr is the expression of its columns, and Expand the
names it selects in a schema:
every := selector.Numeric().Difference(selector.ByName("order_id"))
df, err := sales().Select(every.Expr()).Head(3).Collect(context.Background())
if err != nil {
log.Fatal(err)
}
fmt.Println(df)
shape: (3, 2)
┌───────┬─────────┐
│ qty │ price │
│ Int64 │ Float64 │
├───────┼─────────┤
│ 10 │ 1.5 │
│ 4 │ 1.5 │
│ 7 │ 2.25 │
└───────┴─────────┘
Drop removes columns and Rename renames them, from a map of old names to new:
df, err := sales().
Drop("order_id", "date").
Rename(map[string]string{"qty": "quantity", "price": "unit_price"}).
Head(3).
Collect(context.Background())
if err != nil {
log.Fatal(err)
}
fmt.Println(df)
shape: (3, 4)
┌─────────┬─────────┬──────────┬────────────┐
│ region │ product │ quantity │ unit_price │
│ String │ String │ Int64 │ Float64 │
├─────────┼─────────┼──────────┼────────────┤
│ "north" │ "apple" │ 10 │ 1.5 │
│ "south" │ "apple" │ 4 │ 1.5 │
│ "north" │ "pear" │ 7 │ 2.25 │
└─────────┴─────────┴──────────┴────────────┘
An expression is named after the first column it reads, and Alias gives it one
name. New in 0.6.7: Name() has Polars' name functions, which name each column a
multi-column expression makes: Prefix, Suffix and Map; ToLower and
ToUpper; Replace, of a regular expression or of the text; and Keep, the first
column's name through any alias:
c := ursus.Col
df, err := sales().
Select(
c("qty", "price").Name().Suffix("_x"),
c("region").Name().ToUpper(),
c("qty").Mul(c("price")).Alias("total").Name().Keep(),
).
Head(3).
Collect(context.Background())
if err != nil {
log.Fatal(err)
}
fmt.Println(df)
shape: (3, 4)
┌───────┬─────────┬─────────┬─────────┐
│ qty_x │ price_x │ REGION │ qty │
│ Int64 │ Float64 │ String │ Float64 │
├───────┼─────────┼─────────┼─────────┤
│ 10 │ 1.5 │ "north" │ 15 │
│ 4 │ 1.5 │ "south" │ 6 │
│ 7 │ 2.25 │ "north" │ 15.75 │
└───────┴─────────┴─────────┴─────────┘
For a struct, PrefixFields, SuffixFields and MapFields rename its fields and
leave the column's name, as Polars' name.prefix_fields and the rest do.
New in 0.6.7. InsertColumn(i, column) puts a column at a place, an expression or
Go values, as Polars' insert_column does; a negative place counts from the end, −1
before the last column:
lines := sales().
Select(ursus.Col("order_id"), ursus.Col("qty"), ursus.Col("price")).
Head(3)
revenue := ursus.Col("qty").Mul(ursus.Col("price")).Alias("revenue")
show(lines.InsertColumn(1, revenue))
show(lines.InsertColumn(-1, ursus.Values("note", []string{"gift", "", "rush"})))
shape: (3, 4)
┌──────────┬─────────┬───────┬─────────┐
│ order_id │ revenue │ qty │ price │
│ Int64 │ Float64 │ Int64 │ Float64 │
├──────────┼─────────┼───────┼─────────┤
│ 1 │ 15 │ 10 │ 1.5 │
│ 2 │ 6 │ 4 │ 1.5 │
│ 3 │ 15.75 │ 7 │ 2.25 │
└──────────┴─────────┴───────┴─────────┘
shape: (3, 4)
┌──────────┬───────┬────────┬─────────┐
│ order_id │ qty │ note │ price │
│ Int64 │ Int64 │ String │ Float64 │
├──────────┼───────┼────────┼─────────┤
│ 1 │ 10 │ "gift" │ 1.5 │
│ 2 │ 4 │ "" │ 1.5 │
│ 3 │ 7 │ "rush" │ 2.25 │
└──────────┴───────┴────────┴─────────┘
New in 0.6.7. ReplaceColumn(i, column) puts a column in place of the one at a
place, and DropInPlace(name) gives a column and the frame without it, as Polars'
replace_column and drop_in_place, each a new frame where Polars changes its own.
GetColumnIndex(name) is a column's place, ToSeries(i) the column at one, and
ToStruct(name) the whole frame as one Struct column.
Select can also compute columns, as WithColumns does: see
Expressions. A pattern is explicit: ursus does not read
Col("^qty$") as a regular expression, as Polars does, so a column named with a ^
can still be selected by name.
Rows¶
Filter keeps the rows where every predicate is true; several predicates are and-ed.
IsIn tests membership and IsBetween a range, both ends included unless a Closed
says otherwise:
df, err := sales().
Filter(
ursus.Col("product").IsIn("apple", "fig"),
ursus.Col("qty").IsBetween(4, 10),
).
Collect(context.Background())
if err != nil {
log.Fatal(err)
}
fmt.Println(df)
shape: (3, 6)
┌──────────┬────────────┬─────────┬─────────┬───────┬─────────┐
│ order_id │ date │ region │ product │ qty │ price │
│ Int64 │ Date │ String │ String │ Int64 │ Float64 │
├──────────┼────────────┼─────────┼─────────┼───────┼─────────┤
│ 1 │ 2026-01-03 │ "north" │ "apple" │ 10 │ 1.5 │
│ 2 │ 2026-01-03 │ "south" │ "apple" │ 4 │ 1.5 │
│ 5 │ 2026-01-05 │ "south" │ "fig" │ 9 │ 3 │
└──────────┴────────────┴─────────┴─────────┴───────┴─────────┘
Or, And and Not combine predicates into one:
df, err := sales().
Filter(ursus.Col("region").Eq("east").Or(ursus.Col("price").Ge(3.0))).
Select(ursus.Col("order_id"), ursus.Col("region"), ursus.Col("price")).
Collect(context.Background())
if err != nil {
log.Fatal(err)
}
fmt.Println(df)
shape: (4, 3)
┌──────────┬─────────┬─────────┐
│ order_id │ region │ price │
│ Int64 │ String │ Float64 │
├──────────┼─────────┼─────────┤
│ 4 │ "east" │ 2.25 │
│ 5 │ "south" │ 3 │
│ 6 │ "east" │ 1.5 │
│ 7 │ "north" │ 3 │
└──────────┴─────────┴─────────┘
An aggregate in a predicate is over the whole frame, so a filter can compare each row with the frame: here, the orders larger than the mean, 6.625.
df, err := sales().
Filter(ursus.Col("qty").Gt(ursus.Col("qty").Mean())).
Select(ursus.Col("order_id"), ursus.Col("qty")).
Collect(context.Background())
if err != nil {
log.Fatal(err)
}
fmt.Println(df)
shape: (4, 2)
┌──────────┬───────┐
│ order_id │ qty │
│ Int64 │ Int64 │
├──────────┼───────┤
│ 1 │ 10 │
│ 3 │ 7 │
│ 5 │ 9 │
│ 6 │ 12 │
└──────────┴───────┘
Nulls in a filter¶
A comparison with a null is null, neither true nor false, and Filter keeps only the
rows where its predicate is true, as SQL's WHERE does. So Filter(p) and
Filter(p.Not()) together can miss rows: the ones where p is null.
Remove is the opposite of Filter: it drops the rows where any predicate is true.
It drops the rows where one is null too, so cai, with no score, is in neither:
ctx := context.Background()
scores := ursus.Frame(
ursus.Values("name", []string{"ana", "ben", "cai", "dee"}),
ursus.ValuesNullable("score",
[]int64{90, 40, 0, 75},
[]bool{true, true, false, true}),
)
kept, err := scores.Filter(ursus.Col("score").Ge(50)).Collect(ctx)
if err != nil {
log.Fatal(err)
}
removed, err := scores.Remove(ursus.Col("score").Ge(50)).Collect(ctx)
if err != nil {
log.Fatal(err)
}
missing, err := scores.Filter(ursus.Col("score").IsNull()).Collect(ctx)
if err != nil {
log.Fatal(err)
}
fmt.Println(kept)
fmt.Println(removed)
fmt.Println(missing)
shape: (2, 2)
┌────────┬───────┐
│ name │ score │
│ String │ Int64 │
├────────┼───────┤
│ "ana" │ 90 │
│ "dee" │ 75 │
└────────┴───────┘
shape: (1, 2)
┌────────┬───────┐
│ name │ score │
│ String │ Int64 │
├────────┼───────┤
│ "ben" │ 40 │
└────────┴───────┘
shape: (1, 2)
┌────────┬───────┐
│ name │ score │
│ String │ Int64 │
├────────┼───────┤
│ "cai" │ null │
└────────┴───────┘
Polars' remove keeps those rows
Polars' remove(p) keeps a row where p is null; ursus's drops it, as its
Filter does. Write the null case out to keep it:
Remove(p.And(p.IsNotNull())), or Filter(p.Not().Or(p.IsNull())).
Sorting¶
Sort takes keys, Asc or Desc of an expression, the first key first:
df, err := sales().
Sort(ursus.Asc(ursus.Col("region")), ursus.Desc(ursus.Col("qty"))).
Select(ursus.Col("region"), ursus.Col("product"), ursus.Col("qty")).
Collect(context.Background())
if err != nil {
log.Fatal(err)
}
fmt.Println(df)
shape: (8, 3)
┌─────────┬─────────┬───────┐
│ region │ product │ qty │
│ String │ String │ Int64 │
├─────────┼─────────┼───────┤
│ "east" │ "apple" │ 12 │
│ "east" │ "pear" │ 2 │
│ "north" │ "apple" │ 10 │
│ "north" │ "pear" │ 7 │
│ "north" │ "fig" │ 3 │
│ "south" │ "fig" │ 9 │
│ "south" │ "pear" │ 6 │
│ "south" │ "apple" │ 4 │
└─────────┴─────────┴───────┘
A sort is stable, so rows equal on every key keep their order. Nulls come first in
either direction, unless a key says NullsLast():
df, err := ursus.Frame(
ursus.ValuesNullable("x", []int64{3, 0, 1, 2}, []bool{true, false, true, true}),
).Sort(ursus.Desc(ursus.Col("x")).NullsLast()).Collect(context.Background())
if err != nil {
log.Fatal(err)
}
fmt.Println(df)
The largest rows¶
TopK(k, keys...) keeps the k largest rows by its keys, and BottomK the k smallest,
without sorting the whole frame. Write each key Asc: TopK takes the largest by it,
and a Desc key ranks that column the other way, as Polars' reverse does. Nulls
never make the cut.
df, err := sales().
TopK(3, ursus.Asc(ursus.Col("qty"))).
Select(ursus.Col("order_id"), ursus.Col("product"), ursus.Col("qty")).
Collect(context.Background())
if err != nil {
log.Fatal(err)
}
fmt.Println(df)
shape: (3, 3)
┌──────────┬─────────┬───────┐
│ order_id │ product │ qty │
│ Int64 │ String │ Int64 │
├──────────┼─────────┼───────┤
│ 6 │ "apple" │ 12 │
│ 1 │ "apple" │ 10 │
│ 5 │ "fig" │ 9 │
└──────────┴─────────┴───────┘
Unique rows¶
Unique(names...) keeps the first row of each distinct key, in the rows' order, or of
each distinct row when no names are given:
df, err := sales().
Unique("region").
Select(ursus.Col("region"), ursus.Col("order_id")).
Collect(context.Background())
if err != nil {
log.Fatal(err)
}
fmt.Println(df)
shape: (3, 2)
┌─────────┬──────────┐
│ region │ order_id │
│ String │ Int64 │
├─────────┼──────────┤
│ "north" │ 1 │
│ "south" │ 2 │
│ "east" │ 4 │
└─────────┴──────────┘
UniqueKeep chooses which row a key keeps: KeepFirst, KeepLast, KeepAny, or
KeepNone, which keeps only the keys seen once. New in 0.6.7.
df, err := sales().
UniqueKeep(ursus.KeepLast, "region").
Select(ursus.Col("region"), ursus.Col("order_id")).
Collect(context.Background())
if err != nil {
log.Fatal(err)
}
fmt.Println(df)
shape: (3, 2)
┌─────────┬──────────┐
│ region │ order_id │
│ String │ Int64 │
├─────────┼──────────┤
│ "east" │ 6 │
│ "north" │ 7 │
│ "south" │ 8 │
└─────────┴──────────┘
Slices¶
Head, Tail and Slice(offset, length) take rows by position. A negative offset
counts from the end, as Polars counts it (new in 0.6.7), and Reverse turns the
frame around:
ctx := context.Background()
ids := func(lf *ursus.LazyFrame) []int64 {
df, err := lf.Collect(ctx)
if err != nil {
log.Fatal(err)
}
s, err := df.Column[int64]("order_id")
if err != nil {
log.Fatal(err)
}
return s.Values()
}
fmt.Println("head 3: ", ids(sales().Head(3)))
fmt.Println("tail 2: ", ids(sales().Tail(2)))
fmt.Println("slice 2, 3: ", ids(sales().Slice(2, 3)))
fmt.Println("slice -3, 2:", ids(sales().Slice(-3, 2)))
fmt.Println("reverse: ", ids(sales().Reverse()))
head 3: [1 2 3]
tail 2: [7 8]
slice 2, 3: [3 4 5]
slice -3, 2: [6 7]
reverse: [8 7 6 5 4 3 2 1]
New in 0.6.7. Gather(ctx, indices, nullOnOOB) takes the rows at positions, in
their order, as Polars' gather; a negative position counts from the end, and one
past the end is refused, or with nullOnOOB is a row of nulls. Row(i) is a row's
values as Go values, RowNamed(i) the same by name, and RowWhere(ctx, predicate)
the one row a predicate holds of. Item() is a one-value frame's value and
ItemAt(row, column) any one, as Polars' row and item.
New in 0.6.7. HashRows(seed, columns...) is each row's hash over its columns, every
column when none is named, as Polars' hash_rows: rows equal as a group-by sees them
hash alike. The numbers are ursus's own, not Polars'.
visits := ursus.Frame(
ursus.Values("user", []string{"ada", "ben", "ada"}),
ursus.Values("page", []string{"/home", "/home", "/home"}),
)
show(visits.WithColumns(ursus.HashRows(0).Alias("hash")))
shape: (3, 3)
┌────────┬─────────┬──────────────────────┐
│ user │ page │ hash │
│ String │ String │ Uint64 │
├────────┼─────────┼──────────────────────┤
│ "ada" │ "/home" │ 15332602214641343754 │
│ "ben" │ "/home" │ 17635480945181825081 │
│ "ada" │ "/home" │ 15332602214641343754 │
└────────┴─────────┴──────────────────────┘
WithRowIndex(name, offset) numbers the rows, from offset:
df, err := sales().
Filter(ursus.Col("region").Eq("south")).
WithRowIndex("n", 1).
Select(ursus.Col("n"), ursus.Col("order_id"), ursus.Col("product")).
Collect(context.Background())
if err != nil {
log.Fatal(err)
}
fmt.Println(df)
shape: (3, 3)
┌────────┬──────────┬─────────┐
│ n │ order_id │ product │
│ Uint32 │ Int64 │ String │
├────────┼──────────┼─────────┤
│ 1 │ 2 │ "apple" │
│ 2 │ 5 │ "fig" │
│ 3 │ 8 │ "pear" │
└────────┴──────────┴─────────┘
ursus has no index of its own, as pandas has: row order is a property of a frame, not a label space, and a row index is a column like any other.