Skip to content

Commit d5faf23

Browse files
authored
feat: multistandardfetcher (#19)
1 parent 671c86c commit d5faf23

File tree

20 files changed

+2768
-177
lines changed

20 files changed

+2768
-177
lines changed

docs/specs.md

Lines changed: 68 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ Go Wallet SDK is a modular Go library intended to support the development of m
1414
| `pkg/eventlog` | Ethereum event log parser for ERC20, ERC721, and ERC1155 events. Automatically detects and parses token events with type-safe access to event data, supporting Transfer, Approval, and other standard token events. |
1515
| `pkg/common` | Shared types and constants. Such as canonical chain IDs (e.g., Ethereum Mainnet, Optimism, Arbitrum, BSC, Base). Developers use these values when configuring the SDK or examples. |
1616
| `pkg/contracts/` | Solidity contracts and Go bindings for smart contract interactions. Includes Multicall3, ERC20, ERC721, and ERC1155 contracts with deployment addresses for multiple chains. |
17-
| `examples/` | Demonstrations of SDK usage. Includes `balance-fetcher-web` (a web interface for batch balance fetching), `ethclient‑usage` (an example that exercises the Ethereum client across multiple RPC endpoints), `multiclient3-usage` (demonstrates multicall functionality), and `eventfilter-example` (shows event filtering and parsing capabilities). | |
17+
| `examples/` | Demonstrations of SDK usage. Includes `balance-fetcher-web` (a web interface for batch balance fetching), `ethclient‑usage` (an example that exercises the Ethereum client across multiple RPC endpoints), `multiclient3-usage` (demonstrates multicall functionality), `multistandardfetcher-example` (shows multi-standard balance fetching across all token types), and `eventfilter-example` (shows event filtering and parsing capabilities). | |
1818

1919
## 2. Architecture
2020

@@ -44,9 +44,9 @@ The balance fetcher is designed to efficiently query balances for many addresses
4444
The multicall package is designed to efficiently batch multiple Ethereum contract calls into single transactions using Multicall3. Its design includes:
4545

4646
- **Call Builders** – Provides functions to build calls for different token types (native ETH, ERC20, ERC721, ERC1155). Each builder creates properly encoded call data for the specific contract function.
47-
- **Job Runner System** – Supports both synchronous (`RunSync`) and asynchronous (`ProcessJobRunners`) execution modes. The system automatically chunks large call sets into manageable batches to avoid transaction size limits.
48-
- **Error Handling** – Graceful failure handling with detailed error reporting. Individual call failures don't cause the entire batch to fail, allowing partial results to be processed.
49-
- **Result Processing** – Provides dedicated result processors for each token type that decode the raw return data into appropriate Go types (`*big.Int` for balances).
47+
- **Job-based System**Uses a flexible job system where each job contains a set of calls and a result processing function. Supports both synchronous (`RunSync`) and asynchronous (`RunAsync`) execution modes. The system automatically chunks large call sets into manageable batches to avoid transaction size limits.
48+
- **Error Handling** – Graceful failure handling with detailed error reporting. Individual call failures don't cause the entire batch to fail, allowing partial results to be processed. Each job can have its own error handling strategy.
49+
- **Result Processing**Each job specifies its own result processing function (`CallResultFn`) that decodes the raw return data into appropriate Go types. Provides dedicated result processors for each token type that decode the raw return data into appropriate Go types (`*big.Int` for balances).
5050
- **Chain Support** – Works with any EVM-compatible chain that has Multicall3 deployed, with automatic address resolution based on chain ID.
5151

5252
### 2.4 Ethereum Client Design
@@ -109,8 +109,8 @@ The multicall package provides efficient batching of multiple Ethereum contract
109109

110110
| Function | Purpose | Parameters | Returns |
111111
|----------|---------|------------|---------|
112-
| `RunSync(ctx, jobs, atBlock, caller, batchSize)` | Execute calls synchronously | `ctx`: `context.Context`, `jobs`: `[][]multicall3.IMulticall3Call`, `atBlock`: `*big.Int`, `caller`: `Caller`, `batchSize`: `int` | `[]JobResult` |
113-
| `ProcessJobRunners(ctx, jobRunners, atBlock, caller, batchSize)` | Execute calls asynchronously | `ctx`: `context.Context`, `jobRunners`: `[]JobRunner`, `atBlock`: `*big.Int`, `caller`: `Caller`, `batchSize`: `int` | `void` |
112+
| `RunSync(ctx, jobs, atBlock, caller, batchSize)` | Execute jobs synchronously | `ctx`: `context.Context`, `jobs`: `[]Job`, `atBlock`: `*big.Int`, `caller`: `Caller`, `batchSize`: `int` | `[]JobResult` |
113+
| `RunAsync(ctx, jobs, atBlock, caller, batchSize)` | Execute jobs asynchronously | `ctx`: `context.Context`, `jobs`: `[]Job`, `atBlock`: `*big.Int`, `caller`: `Caller`, `batchSize`: `int` | `<-chan JobsResult` |
114114

115115
#### 3.1.3 Result Processing
116116

@@ -124,16 +124,26 @@ The multicall package provides efficient batching of multiple Ethereum contract
124124
#### 3.1.4 Types
125125

126126
```go
127+
type Job struct {
128+
Calls []multicall3.IMulticall3Call
129+
CallResultFn func(multicall3.IMulticall3Result) (any, error)
130+
}
131+
132+
type CallResult struct {
133+
Value any
134+
Err error
135+
}
136+
127137
type JobResult struct {
128-
Results []multicall3.IMulticall3Result
138+
Results []CallResult
129139
Err error
130140
BlockNumber *big.Int
131141
BlockHash common.Hash
132142
}
133143

134-
type JobRunner struct {
135-
Job []multicall3.IMulticall3Call
136-
ResultCh chan<- JobResult
144+
type JobsResult struct {
145+
JobIdx int
146+
JobResult JobResult
137147
}
138148

139149
type Caller interface {
@@ -494,48 +504,69 @@ func main() {
494504
common.HexToAddress("0x6B175474E89094C44Da98b954EedeAC495271d0F"), // DAI
495505
}
496506

497-
// Build native balance calls
507+
// Create native balance job
498508
nativeCalls := make([]multicall3.IMulticall3Call, 0, len(accounts))
499509
for _, account := range accounts {
500510
nativeCalls = append(nativeCalls, multicall.BuildNativeBalanceCall(account, multicallAddr))
501511
}
512+
nativeJob := multicall.Job{
513+
Calls: nativeCalls,
514+
CallResultFn: func(result multicall3.IMulticall3Result) (any, error) {
515+
return multicall.ProcessNativeBalanceResult(result)
516+
},
517+
}
502518

503-
// Build ERC20 balance calls
519+
// Create ERC20 balance job
504520
tokenCalls := make([]multicall3.IMulticall3Call, 0, len(accounts)*len(tokens))
505521
for _, account := range accounts {
506522
for _, token := range tokens {
507523
tokenCalls = append(tokenCalls, multicall.BuildERC20BalanceCall(account, token))
508524
}
509525
}
526+
tokenJob := multicall.Job{
527+
Calls: tokenCalls,
528+
CallResultFn: func(result multicall3.IMulticall3Result) (any, error) {
529+
return multicall.ProcessERC20BalanceResult(result)
530+
},
531+
}
510532

511-
// Execute calls synchronously
512-
results := multicall.RunSync(ctx, [][]multicall3.IMulticall3Call{nativeCalls, tokenCalls}, nil, caller, 100)
533+
// Execute jobs synchronously
534+
jobs := []multicall.Job{nativeJob, tokenJob}
535+
results := multicall.RunSync(ctx, jobs, nil, caller, 100)
513536

514537
// Process native balance results
515-
for i, result := range results[0].Results {
516-
balance, err := multicall.ProcessNativeBalanceResult(result)
517-
if err != nil {
518-
fmt.Printf("Error processing native balance for account %d: %v\n", i, err)
538+
for i, callResult := range results[0].Results {
539+
if callResult.Err != nil {
540+
fmt.Printf("Error processing native balance for account %d: %v\n", i, callResult.Err)
519541
continue
520542
}
543+
balance := callResult.Value.(*big.Int)
521544
fmt.Printf("Account %s native balance: %s wei\n", accounts[i].Hex(), balance.String())
522545
}
523546

524547
// Process token balance results
525548
callIndex := 0
526549
for _, account := range accounts {
527550
for _, token := range tokens {
528-
result := results[1].Results[callIndex]
529-
balance, err := multicall.ProcessERC20BalanceResult(result)
530-
if err != nil {
531-
fmt.Printf("Error processing token balance: %v\n", err)
551+
callResult := results[1].Results[callIndex]
552+
if callResult.Err != nil {
553+
fmt.Printf("Error processing token balance: %v\n", callResult.Err)
532554
callIndex++
533555
continue
534556
}
557+
balance := callResult.Value.(*big.Int)
535558
fmt.Printf("Account %s token %s balance: %s\n", account.Hex(), token.Hex(), balance.String())
536559
callIndex++
537560
}
538561
}
562+
563+
// Alternative: Execute jobs asynchronously
564+
// resultsCh := multicall.RunAsync(ctx, jobs, nil, caller, 100)
565+
// for result := range resultsCh {
566+
// jobIdx := result.JobIdx
567+
// jobResult := result.JobResult
568+
// // Process results as they become available
569+
// }
539570
}
540571
```
541572

@@ -559,7 +590,21 @@ The `examples/ethclient-usage` folder shows how to use the Ethereum client acros
559590

560591
- **Code Structure** – The example is split into `main.go`, which loops over endpoints, and helper functions such as `testRPC()` that call various methods and handle errors.
561592

562-
### 4.4 Event Filter Example
593+
### 4.4 Multi-Standard Fetcher Example
594+
595+
The `examples/multistandardfetcher-example` folder demonstrates how to use the multistandardfetcher package to fetch balances across all token standards (Native ETH, ERC20, ERC721, ERC1155) for a specific address using Multicall3 batched calls.
596+
597+
- **Features** – The example fetches native ETH balance, queries ERC20 token balances for popular tokens (USDC, DAI, USDT, WBTC, LINK, UNI, MATIC, SHIB), checks ERC721 NFT balances for well-known collections (BAYC, MAYC, CryptoPunks, Azuki, Moonbirds, Doodles), and retrieves ERC1155 collectible balances from popular contracts. It displays results in a formatted report with token symbols and readable balances.
598+
599+
- **Usage** – Users set the `RPC_URL` environment variable and run the example. The program automatically queries vitalik.eth's balances across all token standards and displays a comprehensive report showing native ETH, ERC20 tokens, ERC721 NFTs, and ERC1155 collectibles with non-zero balances.
600+
601+
- **Multi-Standard Support** – Demonstrates the unified interface for fetching balances across different token standards in a single operation, leveraging the underlying multicall package for efficient batching.
602+
603+
- **Output Format** – The example displays a clean, formatted report with sections for each token type, showing token symbols, balances, and summary statistics. It includes proper error handling and graceful degradation when calls fail.
604+
605+
- **Integration** – The example demonstrates the seamless integration between the `multistandardfetcher` and `multicall` packages, showing how to efficiently fetch balances across multiple token standards with minimal RPC calls.
606+
607+
### 4.5 Event Filter Example
563608

564609
The `examples/eventfilter-example` folder demonstrates how to use the event filter and event log parser packages to detect and display transfer events for specific accounts with concurrent processing.
565610

examples/multiclient3-usage/go.mod

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ module github.com/status-im/go-wallet-sdk/examples/multiclient3-usage
33
go 1.23.0
44

55
require (
6-
github.com/ethereum/go-ethereum v1.16.0
6+
github.com/ethereum/go-ethereum v1.16.3
77
github.com/status-im/go-wallet-sdk v0.0.0
88
)
99

examples/multiclient3-usage/go.sum

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,8 @@ github.com/crate-crypto/go-ipa v0.0.0-20240724233137-53bbb0ceb27a h1:W8mUrRp6NOV
3636
github.com/crate-crypto/go-ipa v0.0.0-20240724233137-53bbb0ceb27a/go.mod h1:sTwzHBvIzm2RfVCGNEBZgRyjwK40bVoun3ZnGOCafNM=
3737
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
3838
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
39+
github.com/dchest/siphash v1.2.3 h1:QXwFc8cFOR2dSa/gE6o/HokBMWtLUaNDVd+22aKHeEA=
40+
github.com/dchest/siphash v1.2.3/go.mod h1:0NvQU092bT0ipiFN++/rXm69QG9tVxLAlQHIXMPAkHc=
3941
github.com/deckarep/golang-set/v2 v2.6.0 h1:XfcQbWM1LlMB8BsJ8N9vW5ehnnPVIw0je80NsVHagjM=
4042
github.com/deckarep/golang-set/v2 v2.6.0/go.mod h1:VAky9rY/yGXJOLEDv3OMci+7wtDpOF4IN+y82NBOac4=
4143
github.com/decred/dcrd/crypto/blake256 v1.0.0 h1:/8DMNYp9SGi5f0w7uCm6d6M4OU2rGFK09Y2A4Xv7EE0=
@@ -44,14 +46,16 @@ github.com/decred/dcrd/dcrec/secp256k1/v4 v4.0.1 h1:YLtO71vCjJRCBcrPMtQ9nqBsqpA1
4446
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.0.1/go.mod h1:hyedUtir6IdtD/7lIxGeCxkaw7y45JueMRL4DIyJDKs=
4547
github.com/deepmap/oapi-codegen v1.6.0 h1:w/d1ntwh91XI0b/8ja7+u5SvA4IFfM0UNNLmiDR1gg0=
4648
github.com/deepmap/oapi-codegen v1.6.0/go.mod h1:ryDa9AgbELGeB+YEXE1dR53yAjHwFvE9iAUlWl9Al3M=
49+
github.com/emicklei/dot v1.6.2 h1:08GN+DD79cy/tzN6uLCT84+2Wk9u+wvqP+Hkx/dIR8A=
50+
github.com/emicklei/dot v1.6.2/go.mod h1:DeV7GvQtIw4h2u73RKBkkFdvVAz0D9fzeJrgPW6gy/s=
4751
github.com/ethereum/c-kzg-4844/v2 v2.1.0 h1:gQropX9YFBhl3g4HYhwE70zq3IHFRgbbNPw0Shwzf5w=
4852
github.com/ethereum/c-kzg-4844/v2 v2.1.0/go.mod h1:TC48kOKjJKPbN7C++qIgt0TJzZ70QznYR7Ob+WXl57E=
49-
github.com/ethereum/go-ethereum v1.16.0 h1:Acf8FlRmcSWEJm3lGjlnKTdNgFvF9/l28oQ8Q6HDj1o=
50-
github.com/ethereum/go-ethereum v1.16.0/go.mod h1:ngYIvmMAYdo4sGW9cGzLvSsPGhDOOzL0jK5S5iXpj0g=
53+
github.com/ethereum/go-ethereum v1.16.3 h1:nDoBSrmsrPbrDIVLTkDQCy1U9KdHN+F2PzvMbDoS42Q=
54+
github.com/ethereum/go-ethereum v1.16.3/go.mod h1:Lrsc6bt9Gm9RyvhfFK53vboCia8kpF9nv+2Ukntnl+8=
5155
github.com/ethereum/go-verkle v0.2.2 h1:I2W0WjnrFUIzzVPwm8ykY+7pL2d4VhlsePn4j7cnFk8=
5256
github.com/ethereum/go-verkle v0.2.2/go.mod h1:M3b90YRnzqKyyzBEWJGqj8Qff4IDeXnzFw0P9bFw3uk=
53-
github.com/ferranbt/fastssz v0.1.2 h1:Dky6dXlngF6Qjc+EfDipAkE83N5I5DE68bY6O0VLNPk=
54-
github.com/ferranbt/fastssz v0.1.2/go.mod h1:X5UPrE2u1UJjxHA8X54u04SBwdAQjG2sFtWs39YxyWs=
57+
github.com/ferranbt/fastssz v0.1.4 h1:OCDB+dYDEQDvAgtAGnTSidK1Pe2tW3nFV40XyMkTeDY=
58+
github.com/ferranbt/fastssz v0.1.4/go.mod h1:Ea3+oeoRGGLGm5shYAeDgu6PGUlcvQhE2fILyD9+tGg=
5559
github.com/fsnotify/fsnotify v1.6.0 h1:n+5WquG0fcWoWp6xPWfHdbskMCQaFnG6PfBrh1Ky4HY=
5660
github.com/fsnotify/fsnotify v1.6.0/go.mod h1:sl3t1tCWJFWoRz9R8WJCbQihKKwmorjAbSClcnxKAGw=
5761
github.com/gballet/go-libpcsclite v0.0.0-20190607065134-2772fd86a8ff h1:tY80oXqGNY4FhTFhk+o9oFHGINQ/+vhlm8HFzi6znCI=
@@ -65,8 +69,8 @@ github.com/gofrs/flock v0.12.1 h1:MTLVXXHf8ekldpJk3AKicLij9MdwOWkZ+a/jHHZby9E=
6569
github.com/gofrs/flock v0.12.1/go.mod h1:9zxTsyu5xtJ9DK+1tFZyibEV7y3uwDxPPfbxeeHCoD0=
6670
github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
6771
github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
68-
github.com/golang-jwt/jwt/v4 v4.5.1 h1:JdqV9zKUdtaa9gdPlywC3aeoEsR681PlKC+4F5gQgeo=
69-
github.com/golang-jwt/jwt/v4 v4.5.1/go.mod h1:m21LjoU+eqJr34lmDMbreY2eSTRJ1cv77w39/MY0Ch0=
72+
github.com/golang-jwt/jwt/v4 v4.5.2 h1:YtQM7lnr8iZ+j5q71MGKkNw9Mn7AjHM68uc9g5fXeUI=
73+
github.com/golang-jwt/jwt/v4 v4.5.2/go.mod h1:m21LjoU+eqJr34lmDMbreY2eSTRJ1cv77w39/MY0Ch0=
7074
github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
7175
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
7276
github.com/golang/snappy v0.0.5-0.20220116011046-fa5810519dcb h1:PBC98N2aIaM3XXiurYmW7fx4GZkL8feAMVq7nEjURHk=

0 commit comments

Comments
 (0)