Commit Graph

553 Commits

Author SHA1 Message Date
Chenna Keshava B S
77c0a62a74 fix: remove redundant moves (#4565)
- Resolve gcc compiler warning:
      AccountObjects.cpp:182:47: warning: redundant move in initialization [-Wredundant-move]
  - The std::move() operation on trivially copyable types may generate a
    compile warning in newer versions of gcc.
- Remove extraneous header (unused imports) from a unit test file.
2023-06-09 17:33:28 -07:00
Shawn Xie
3620ac287e Add nftoken_id, nftoken_ids, offer_id fields for NFTokens (#4447)
Three new fields are added to the `Tx` responses for NFTs:

1. `nftoken_id`: This field is included in the `Tx` responses for
   `NFTokenMint` and `NFTokenAcceptOffer`. This field indicates the
   `NFTokenID` for the `NFToken` that was modified on the ledger by the
   transaction.
2. `nftoken_ids`: This array is included in the `Tx` response for
   `NFTokenCancelOffer`. This field provides a list of all the
   `NFTokenID`s for the `NFToken`s that were modified on the ledger by
   the transaction.
3. `offer_id`: This field is included in the `Tx` response for
   `NFTokenCreateOffer` transactions and shows the OfferID of the
   `NFTokenOffer` created.

The fields make it easier to track specific tokens and offers. The
implementation includes code (by @ledhed2222) from the Clio project to
extract NFTokenIDs from mint transactions.
2023-05-18 16:38:18 -07:00
drlongle
78076a6903 fix!: Prevent API from accepting seed or public key for account (#4404)
The API would allow seeds (and public keys) to be used in place of
accounts at several locations in the API. For example, when calling
account_info, you could pass `"account": "foo"`. The string "foo" is
treated like a seed, so the method returns `actNotFound` (instead of
`actMalformed`, as most developers would expect). In the early days,
this was a convenience to make testing easier. However, it allows for
poor security practices, so it is no longer a good idea. Allowing a
secret or passphrase is now considered a bug. Previously, it was
controlled by the `strict` option on some methods. With this commit,
since the API does not interpret `account` as `seed`, the option
`strict` is no longer needed and is removed.

Removing this behavior from the API is a [breaking
change](https://xrpl.org/request-formatting.html#breaking-changes). One
could argue that it shouldn't be done without bumping the API version;
however, in this instance, there is no evidence that anyone is using the
API in the "legacy" way. Furthermore, it is a potential security hole,
as it allows users to send secrets to places where they are not needed,
where they could end up in logs, error messages, etc. There's no reason
to take such a risk with a seed/secret, since only the public address is
needed.

Resolves: #3329, #3330, #4337

BREAKING CHANGE: Remove non-strict account parsing (#3330)
2023-05-16 17:22:10 -07:00
RichardAH
f191c911d4 Add NFTokenPages to account_objects RPC: (#4352)
- Include NFTokenPages in account_objects to make it easier to
  understand an account's Owner Reserve and simplify app development.
- Update related tests and documentation.
- Fix #4347.

For info about the Owner Reserve, see https://xrpl.org/reserves.html

---------

Co-authored-by: Scott Schurr <scott@ripple.com>
Co-authored-by: Ed Hennis <ed@ripple.com>
2023-04-05 13:58:55 -07:00
drlongle
2f3f6dcb03 Fix ledger_data to return an empty list: (#4398)
Change `ledger_data` to return an empty list when all entries are
filtered out.

When the `type` field is specified for the `ledger_data` method, it is
possible that no objects of the specified type are found. This can even
occur if those objects exist, but not in the section that the server
checked while serving your request. Previously, the `state` field of the
response has the value `null`, instead of an empty array `[]`. By
changing this to an empty array, the response is the same data type so
that clients can handle it consistently.

For example, in Python, `for entry in state` should now work correctly.
It would raise an exception if `state` is `null` (or `None`). 

This could break client code that explicitly checks for null. However,
this fix aligns the response with the documentation, where the `state`
field is an array.

Fix #4392.
2023-03-30 11:59:10 -07:00
drlongle
5ebcaf0a6c Add account flags to account_info response: (#4459)
Previously, the object `account_data` in the `account_info` response
contained a single field `Flags` that contains flags of an account. API
consumers must perform bitwise operations on this field to retrieve the
account flags.

This change adds a new object, `account_flags`, at the top level of the
`account_info` response `result`. The object contains relevant flags of
the account. This makes it easier to write simple code to check a flag's
value.

The flags included may depend on the amendments that are enabled.

Fix #2457.
2023-03-30 11:46:18 -07:00
drlongle
8bfdbcbab5 Add logging for exceptions: (#4400)
Log exception messages at several locations.

Previously, these were locations where an exception was caught, but the
exception message was not logged. Logging the exception messages can be
useful for analysis or debugging. The additional logging could have a
small negative performance impact.

Fix #3213.
2023-03-30 10:13:30 -07:00
Ed Hennis
66627b26cf Refactor fee initialization and configuration: (#4319)
* Create the FeeSettings object in genesis ledger.
* Initialize with default values from the config. Removes the need to
  pass a Config down into the Ledger initialization functions, including
  setup().
* Drop the undocumented fee config settings in favor of the [voting]
  section.
  * Fix #3734.
  * If you previously used fee_account_reserve and/or fee_owner_reserve,
    you should change to using the [voting] section instead. Example:

```
[voting]
account_reserve=10000000
owner_reserve=2000000
```

* Because old Mainnet ledgers (prior to 562177 - yes, I looked it up)
  don't have FeeSettings, some of the other ctors will default them to
  the config values before setup() tries to load the object.
* Update default Config fee values to match Mainnet.
* Fix unit tests:
  * Updated fees: Some tests are converted to use computed values of fee
    object, but the default Env config was also updated to fix the rest.
  * Unit tests that check the structure of the ledger have updated
    hashes and counts.
2023-03-28 09:03:25 -07:00
Ed Hennis
9b2d563dec fix: support RPC markers for any ledger object: (#4361)
There were situations where `marker`s  returned by `account_lines` did
not work on subsequent requests, returning "Invalid Parameters".

This was caused by the optimization implemented in "Enforce account RPC
limits by account objects traversed":

e28989638d

Previously, the ledger traversal would find up to `limit` account lines,
and if there were more, the marker would be derived from the key of the
next account line. After the change, ledger traversal would _consider_
up to `limit` account objects of any kind found in the account's
directory structure. If there were more, the marker would be derived
from the key of the next object, regardless of type.

With this optimization, it is expected that `account_lines` may return
fewer than `limit` account lines - even 0 - along with a marker
indicating that there are may be more available.

The problem is that this optimization did not update the
`RPC::isOwnedByAccount` helper function to handle those other object
types. Additionally, XLS-20 added `ltNFTOKEN_OFFER` ledger objects to
objects that have been added to the account's directory structure, but
did not update `RPC::isOwnedByAccount` to be able to handle those
objects. The `marker` provided in the example for #4354 includes the key
for an `ltNFTOKEN_OFFER`. When that `marker` is used on subsequent
calls, it is not recognized as valid, and so the request fails.

* Add unit test that walks all the object types and verifies that all of
  their indexes can work as a marker.
* Fix #4340
* Fix #4354
2023-03-20 10:22:15 -07:00
RichardAH
10555faa92 fix(gateway_balances): handle overflow exception: (#4355)
* Prevent internal error by catching overflow exception in `gateway_balances`.
* Treat `gateway_balances` obligations overflow as max (largest valid) `STAmount`.
  * Note that very large sums of STAmount are approximations regardless.

---------

Co-authored-by: Scott Schurr <scott@ripple.com>
2023-03-16 10:25:40 -07:00
RichardAH
cb08f2b6ec Allow port numbers be be specified with a colon: (#4328)
Port numbers can now be specified using either a colon or a space.

Examples:

1.2.3.4:51235

1.2.3.4 51235

- In the configuration file, an annoying "gotcha" for node operators is
  accidentally specifying IP:PORT combinations using a colon. The code
  previously expected a space, not a colon. It also does not provide
  good feedback when this operator error is made.
- This change simply allows this mistake (using a colon) to be fixed
  automatically, preserving the intention of the operator.
- Add unit tests, which test the functionality when specifying IP:PORT
  in the configuration file.
- The RPCCall test regime is not specific enough to test this
  functionality, it has been tested by hand.
- Ensure IPv6 addresses are not confused for ip:port

---------

Co-authored-by: Elliot Lee <github.public@intelliot.com>
2023-03-14 21:06:30 -07:00
Howard Hinnant
c9c54c9799 Add implicit conversion from STAmount to Number 2023-02-07 15:43:28 -08:00
Ed Hennis
e4b17d1cf2 XRPFees: Fee setting and handling improvements (#4247)
* Introduces amendment `XRPFees`
* Convert fee voting and protocol messages to use XRPAmounts
* Includes Validations, Change transactions, the "Fees" ledger object,
  and subscription messages

* Improve handling of 0 drop reference fee with TxQ. For use with networks that do not want to require fees
* Note that fee escalation logic is still in place, which may cause the
  open ledger fee to rise if the network is busy. 0 drop transactions
  will still queue, and fee escalation can be effectively disabled by
  modifying the configuration on all nodes

* Change default network reserves to match Mainnet

* Name the new SFields *Drops (not *XRP)
* Reserve SField IDs for Hooks

* Clarify comments explaining the ttFEE transaction field validation
2023-02-02 16:20:35 -08:00
Scott Schurr
6f8750316c RPC tooBusy response has 503 HTTP status if "ripplerpc": "3.0": (#4143)
Fixes #4005

Makes it possible for internal RPC Error Codes to associate
themselves with a non-OK (200) HTTP status code.  There are
quite a number of RPC responses in addition to tooBusy that
now have non-OK HTTP status codes.

The new return HTTP return codes are only enabled by including
"ripplerpc": "3.0" or higher in the original request.
Otherwise the historical value, 200, continues to be returned.
This ensures that this is not a breaking change.
2023-01-03 09:24:45 -08:00
John Freeman
c3a9f3dbf3 Use the Conan package manager (#4367)
Introduces a conanfile.py (and a Conan recipe for RocksDB) to enable building the package with Conan, choosing more recent default versions of dependencies. It removes almost all of the CMake build files related to dependencies, and the configurations for Travis CI and GitLab CI. A new set of cross-platform build instructions are written in BUILD.md.

Includes example GitHub Actions workflow for each of Linux, macOS, Windows.

* Test on macos-12

We use the <concepts> library which was not added to Apple Clang until
version 13.1.6. The default Clang on macos-11 (the sometimes current
version of macos-latest) is 13.0.0, and the default Clang on macos-12 is
14.0.0.

Closes #4223.
2022-12-16 10:46:22 -08:00
CJ Cobb
df1300fb37 Catch transaction deserialization error in doLedgerGrpc (#4323)
* Allow clio to extract ledgers with transactions that can no longer be
  deserialized. The problem transactions will be skipped.
2022-12-13 17:09:53 -08:00
Scott Determan
648d6c3e2f Fix clang 15 warnings: (#4325)
Clang warned about the code removed in this patch with the warning:
```
warning: out-of-line definition of constexpr static data member is
redundant in C++17 and is deprecated [-Wdeprecated]
```
2022-12-13 16:21:22 -08:00
CJ Cobb
28f4cc7817 Remove gRPC code previously used for the xpring SDK 2022-10-13 15:44:43 -07:00
Alexander Kremer
23ce431876 Remove const_cast usage 2022-10-11 16:38:10 -07:00
Nik Bougalis
e2eed966b0 Improve AccountID string conversion caching:
Caching the base58check encoded version of an `AccountID` has
performance advantages, because because of the computationally
heavy cost associated with the conversion, which requires the
application of SHA-256 twice.

This commit makes the cache significantly more efficient in terms
of memory used: it eliminates the map, using a vector with a size
that is determined by the configured size of the node, and a hash
function to directly map any given `AccountID` to a specific slot
in the cache; the eviction policy is simple: in case of collision
the existing entry is removed and replaced with the new data.

Previously, use of the cache was optional and required additional
effort by the programmer. Now the cache is automatic and does not
require any additional work or information.

The new cache also utilizes a 64-way spinlock, to help reduce any
contention that the pressure on the cache would impose.
2022-08-25 08:49:16 -07:00
Chenna Keshava B S
ce64f7a90f Remove deprecated AccountTxOld.cpp (fixes #2926) 2022-08-25 08:49:04 -07:00
seelabs
92d35e54c7 Switch from C++17 to C++20 2022-08-25 08:38:28 -07:00
Nik Bougalis
b95ca98965 Fix #4231 (wrong include guard) and an out-of-order construction warning 2022-07-17 22:17:34 -07:00
Nathan Nichols
d458e9972b Improve JSON sanitization in reporting mode 2022-07-17 22:17:24 -07:00
RichardAH
610436d737 Add book_changes rpc (#4212) 2022-07-11 09:29:31 +02:00
Scott Schurr
80bda7cc48 Introduce fixNFTokenDirV1 amendment:
o Fixes an off-by-one when determining which NFTokenPage an
  NFToken belongs on.
o Improves handling of packed sets of 32 NFTs with
  identical low 96-bits.
o Fixes marker handling by the account_nfts RPC command.
o Tightens constraints of NFTokenPage invariant checks.

Adds unit tests to exercise the fixed cases as well as tests
for previously untested functionality.
2022-05-10 13:34:12 -07:00
Devon White
dac080f1c8 Advance ripple.app.rdb 2022-05-10 13:34:12 -07:00
Richard Holland
01c37fed69 Introduce the ExpandedSignerList amendment:
The amendment increases the maximum sign of an account's signer
list from 8 to 32.

Like all new features, the associated amendment is configured with
a default vote of "no" and server operators will have to vote for
it explicitly if they believe it is useful.
2022-05-10 13:34:12 -07:00
greg7mdp
dfe69f1b76 Fix strict aliasing test failures with newer MSVC compilers 2022-05-10 13:34:12 -07:00
Nik Bougalis
70779f6850 Introduce NFT support (XLS020) 2022-04-06 13:29:48 -07:00
Nik Bougalis
8f58687091 Cleanup AcceptedLedger and AcceptedLedgerTx:
This commit modernizes the `AcceptedLedger` and `AcceptedLedgerTx`
classes, reduces their memory footprint and reduces unnecessary
dynamic memory allocations.
2022-03-30 15:16:27 -07:00
Nik Bougalis
c7e6803956 Simplify LedgerMaster:
- Eliminate `tune` member function and allow `LedgerHistory`
  to fully initialize itself.
2022-03-30 15:16:27 -07:00
Nik Bougalis
34ca457132 Improve stop signaling for Application 2022-03-29 16:15:42 -07:00
Edward Hennis
df60e46750 Improve deterministic transaction sorting in TxQ:
* Txs with the same fee level will sort by TxID XORed with the parent
  ledger hash.
* The TxQ is re-sorted after every ledger.
* Attempt to future-proof the TxQ tie breaking test
2022-03-23 23:28:04 -07:00
Edward Hennis
e7e672c3f8 Incremental improvements to path finding memory usage:
* Abort background path finding when closed or disconnected
* Exit pathfinding job thread if there are no requests left
* Don't bother creating the path find job if there are no requests
* Refactor to remove circular dependency between InfoSub and PathRequest
2022-03-23 23:28:04 -07:00
seelabs
4d5459d041 Optimize trust line caching:
The existing trust line caching code was suboptimal in that it stored
redundant information, pinned SLEs into memory and required multiple
memory allocations per cached object.

This commit eliminates redundant data, reducing the size of cached
objects and unpinning SLEs from memory, and uses value_types to
avoid the need for `std::shared_ptr`. As a result of these changes, the
effective size of a cached object, includes the overhead of the memory
allocator and the `std::shared_ptr` should be reduced by at least 64
bytes. This is significant, as there can easily be tens of millions
of these objects.
2022-03-23 23:28:04 -07:00
CJ Cobb
11ca9a946c Add successor information to clio ETL messages
* Allow clio to ask for object successors and predecessors from rippled
* Add lower_bound and last_below to SHAMap
2022-03-04 13:38:48 -08:00
natenichols
90326bf756 Proxy validation_quorum when in reporting mode 2022-03-04 13:38:48 -08:00
Mark Travis
8f82b62e0d Use CIDR notation for admin and secure_gateway 2022-03-01 11:27:50 -08:00
Mark Travis
dc213a4fab Make gateway_balances admin-only in reporting mode 2022-03-01 11:27:34 -08:00
John Freeman
c2a08a1f26 Simplify the Job Queue:
This is a refactor aimed at cleaning up and simplifying the existing
job queue.

As of now, all jobs are cancelled at the same time and in the same
way, so this commit removes the per-job cancellation token. If the
need for such support is demonstrated, support can be re-added.

* Revise documentation for ClosureCounter and Workers.
* Simplify code, removing unnecessary function arguments and
  deduplicating expressions
* Restructure job handlers to no longer need to pass a job's
  handle to the job.
2022-03-01 11:25:03 -08:00
natenichols
e28989638d Enforce account RPC limits by account objects traversed 2021-12-16 11:57:08 -08:00
CJ Cobb
915fe31274 log request and duration for every RPC call 2021-12-15 12:12:28 -08:00
Mark Travis
db720a59e4 Log resource limit disconnections. 2021-12-15 11:26:13 -08:00
Scott Schurr
d54f6278bb Improve names returned by server_info counters 2021-12-15 11:21:51 -08:00
Scott Schurr
ad805eb95b Improve handling of the X-Forwarded-For and Forwarded HTTP headers 2021-12-14 17:47:04 -08:00
Richard Holland
daccb5b4c0 Add ignore_default option to account_lines API:
This flag, if present, suppresses the output of incoming
trustlines in default state.

This is primarily motivated by observing that users of Xumm often
have many unwanted incoming trustlines in a default state, which are
not useful in the vast majority of cases.

Being able to suppress those when doing `account_lines` saves bandwidth
and resources.
2021-12-14 17:44:24 -08:00
Edward Hennis
5050b366d9 Return correct error code for node_to_shard command errors 2021-11-22 18:40:32 -08:00
Peng Wang
e2a42184b9 Implement tx stream with history 2021-10-21 12:05:48 -07:00
Devon White
00a4c3a478 Implement node-to-shard RPC control 2021-10-19 16:27:30 -07:00