You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This PR contains some refactoring and a new `record` column that allows
you to use a `BinaryMarshaler` and `BinaryUnmarshaler` type to be
stored. As such, it supports types that implement this standard way of
encoding, for example `time.Time`.
```go
col := NewCollection()
col.CreateColumn("timestamp", ForRecord(func() *time.Time {
return new(time.Time)
}, nil))
// Insert the time, it implements binary marshaler
idx, _ := col.Insert(func(r Row) error {
now := time.Unix(1667745766, 0)
r.SetRecord("timestamp", &now)
return nil
})
// We should be able to read back the time
col.QueryAt(idx, func(r Row) error {
ts, ok := r.Record("timestamp")
assert.True(t, ok)
assert.Equal(t, "November", ts.(*time.Time).UTC().Month().String())
return nil
})
```
In order to get data into the store, you'll need to first create a `Collection` by calling `NewCollection()` method. Each collection requires a schema, which can be either specified manually by calling `CreateColumn()` multiple times or automatically inferred from an object by calling `CreateColumnsOf()` function.
52
-
53
-
In the example below we're loading some `JSON` data by using `json.Unmarshal()` and auto-creating colums based on the first element on the loaded slice. After this is done, we can then load our data by inserting the objects one by one into the collection. This is accomplished by calling `InsertObject()` method on the collection itself repeatedly.
52
+
In order to get data into the store, you'll need to first create a `Collection` by calling `NewCollection()` method. Each collection requires a schema, which needs to be specified by calling `CreateColumn()` multiple times or automatically inferred from an object by calling `CreateColumnsOf()` function. In the example below we create a new collection with several columns.
54
53
55
54
```go
56
-
data:=loadFromJson("players.json")
57
-
58
-
// Create a new columnar collection
59
-
players:= column.NewCollection()
60
-
players.CreateColumnsOf(data[0])
61
-
62
-
// Insert every item from our loaded data
63
-
for_, v:=range data {
64
-
players.InsertObject(v)
65
-
}
66
-
```
67
-
68
-
Now, let's say we only want specific columns to be added. We can do this by calling `CreateColumn()` method on the collection manually to create the required columns.
69
-
70
-
```go
71
-
// Create a new columnar collection with pre-defined columns
Now that we have created a collection, we can insert a single record by using `Insert()` method on the collection. In this example we're inserting a single row and manually specifying values. Note that this function returns an `index` that indicates the row index for the inserted row.
While the previous example demonstrated how to insert many objects, it was doing it one by one and is rather inefficient. This is due to the fact that each `InsertObject()` call directly on the collection initiates a separate transacion and there's a small performance cost associated with it. If you want to do a bulk insert and insert many values, faster, that can be done by calling `Insert()` on a transaction, as demonstrated in the example below. Note that the only difference is instantiating a transaction by calling the `Query()` method and calling the `txn.Insert()` method on the transaction instead the one on the collection.
75
+
While the previous example demonstrated how to insert a single row, inserting multiple rows this way is rather inefficient. This is due to the fact that each `Insert()` call directly on the collection initiates a separate transacion and there's a small performance cost associated with it. If you want to do a bulk insert and insert many values, faster, that can be done by calling `Insert()` on a transaction, as demonstrated in the example below. Note that the only difference is instantiating a transaction by calling the `Query()` method and calling the `txn.Insert()` method on the transaction instead the one on the collection.
If you find yourself in need of encoding a more complex structure as a single column, you may do so by using `column.ForRecord()` function. This allows you to specify a `BinaryMarshaler` / `BinaryUnmarshaler` type that will get automatically encoded as a single column. In th example below we are creating a `Location` type that implements the required methods.
In order to manipulate the record, we can use the appropriate `Record()`, `SetRecord()` methods of the `Row`, similarly to other column types.
378
+
379
+
```go
380
+
// Insert a new location
381
+
idx, _:= players.Insert(func(r Row) error {
382
+
r.SetRecord("location", &Location{X: 1, Y: 2})
383
+
returnnil
384
+
})
385
+
386
+
// Read the location back
387
+
players.QueryAt(idx, func(r Row) error {
388
+
location, ok:= r.Record("location")
389
+
returnnil
390
+
})
391
+
```
392
+
359
393
## Streaming Changes
360
394
361
395
This library also supports streaming out all transaction commits consistently, as they happen. This allows you to implement your own change data capture (CDC) listeners, stream data into kafka or into a remote database for durability. In order to enable it, you can simply provide an implementation of a `commit.Logger` interface during the creation of the collection.
Multiple complete usage examples of this library can be found in the [examples](https://github.com/kelindar/column/tree/main/examples) directory in this repository.
0 commit comments