Skip to main content
日本語 中文 (新加坡)

Search

Articles in this section

See more

Get to know your node logs

Avatar
Carlos
  • Updated

Daedalus is a full node wallet. This means that there is a cardano-node running under the hood on your machine when you use Daedalus. Understanding the logs from the node can be helpful, in particular when there is an issue with your wallet.

 

To see your logs in real time:

  1. Open PowerShell (Windows) or a Terminal (macOS and Linux) 
  2. Copy/paste the following command: 
    • on Windows
       
      Get-Content -tail 0  .\AppData\Roaming\”Daedalus Mainnet”\Logs\pub\node.log -Wait

     

    • on macOS
       
      tail -fn 0 ~/Library/Application\ Support/Daedalus\ Mainnet/Logs/pub/node.log 

     

    • on Linux
       
      tail -fn 0 ~/.local/share/Daedalus/mainnet/Logs/pub/node.log
  3. Prompt will blink until you
  4. Open Daedalus

 

This is what a well-functioning node looks like:

1. Node initialization

When you launch Daedalus, logs show the network you are connecting to, node version, node startup time, and blockchain parameters:

[cardano.node.networkMagic:Notice:5] [Date Time] NetworkMagic 764824073
[cardano.node.basicInfo.protocol:Notice:5] [Date Time] Byron; Shelley
[cardano.node.basicInfo.protocol:Notice:5] [Date Time] 1.27.0
[cardano.node.basicInfo.protocol:Notice:5] [Date Time] 8fe46140a52810b6ca456be01d652ca08fe730bf
[cardano.node.basicInfo.nodeStartTime:Notice:5] [Date Time] 2021-09-07 02:36:29.124891 UTC
[cardano.node.basicInfo.systemStartTime:Notice:5] [Date Time] 2017-09-23 21:44:51 UTC
[cardano.node.basicInfo.slotLengthByron:Notice:5] [Date Time] 20s
[cardano.node.basicInfo.epochLengthByron:Notice:5] [Date Time] 21600
[cardano.node.basicInfo.slotLengthShelley:Notice:5] [Date Time] 1s
[cardano.node.basicInfo.epochLengthShelley:Notice:5] [Date Time] 432000
[cardano.node.basicInfo.slotsPerKESPeriodShelley:Notice:5] [Date Time] 129600
[cardano.node.basicInfo.slotLengthAllegra:Notice:5] [Date Time] 1s
[cardano.node.basicInfo.epochLengthAllegra:Notice:5] [Date Time] 432000
[cardano.node.basicInfo.slotsPerKESPeriodAllegra:Notice:5] [Date Time] 129600
[cardano.node.basicInfo.slotLengthMary:Notice:5] [Date Time] 1s
[cardano.node.basicInfo.epochLengthMary:Notice:5] [Date Time] 432000
[cardano.node.basicInfo.slotsPerKESPeriodMary:Notice:5] [Date Time] 129600

2. Network setup

Then, the node gets ready to connect to the network: sets a port for the node socket, the diffusion mode, and gets all the IPs or DNS of the nodes it will attempt to connect to.

[cardano.node.addresses:Notice:5] ... [SocketInfo 0.0.0.0:59155,SocketInfo [::]:59155]
[cardano.node.diffusion-mode:Notice:5] ... InitiatorAndResponderDiffusionMode
[cardano.node.dns-producers:Notice:5] ... [DnsSubscriptionTarget {dstDomain = "relays-new.cardano-mainnet.iohk.io", dstPort = 3001, dstValency = 1}]
[cardano.node.ip-producers:Notice:5] ... IPSubscriptionTarget {ispIps = [], ispValency = 0}

3. Construct LedgerState and opening databases

Before starting any connection attempts, the node performs a resource-intensive and time-consuming task: constructing a LedgerState that corresponds to the LedgerState at the tip of the ImmutableDB (the part of the blockchain that cannot change.) This task can take from a few minutes to several hours, depending on many factors such as: Database state, when the last time it was in sync, whether it was closed cleanly or not,  CPU and RAM. This process is not logged at the default configuration level, but logs show when the databases are finally opened. 

The node opens the immutable DB, volatileDB, and LedgerDB: 

[cardano.node.ChainDB:Info:31] ... Opened imm db with immutable tip at
  a6c63f8f8230ad0aaa1d9086655a4d10740e94d18ae34584169e95dac78065b0 at slot 39371215 and chunk 1822
[cardano.node.ChainDB:Info:31] ... Opened vol db
[cardano.node.ChainDB:Info:31] ... Replaying ledger from snapshot DiskSnapshot {dsNumber = 39367150, dsSuffix = Nothing}
  at f81ff147dd36ab19a63b58c70e2cad3764c309579431655da347a046d9cb108a at slot 39367150
[cardano.node.ChainDB:Info:31] ... Replayed block: slot SlotNo 39367187 of At (SlotNo 39371215)
[cardano.node.ChainDB:Info:31] ... before next, messages elided = 39367224
[cardano.node.ChainDB:Info:31] ... Replayed block: slot SlotNo 39371215 of At (SlotNo 39371215)
[cardano.node.ChainDB:Info:31] ... Opened lgr db
[cardano.node.ChainDB:Info:31] ... Opened db with immutable tip at 
  a6c63f8f8230ad0aaa1d9086655a4d10740e94d18ae34584169e95dac78065b0 at slot 39371215

3. Connection attempt

Once the node has a LedgerState ready and databases opened, it is ready to establish connections with other nodes and update its copy of the blockchain. By default, Daedalus is configured to connect to the relays at relays-new.cardano-mainnet.iohk.io We have several servers behind this DNS, the node attempts to connect to all of them, when it establishes a good connection with one of the relay nodes, you see ConnectSuccessLast and the connection attempts to the other relays are cancelled SubscriberParallelConnectionCancelled

[cardano.node.DnsSubscription:Notice:73] ... Domain: "relays-new.cardano-mainnet.iohk.io" 
  Connection Attempt Start, destination 18.223.202.44:3001
[cardano.node.DnsSubscription:Notice:74] ... Domain: "relays-new.cardano-mainnet.iohk.io" 
  Connection Attempt Start, destination 18.157.126.47:3001
[cardano.node.DnsSubscription:Notice:75] ... Domain: "relays-new.cardano-mainnet.iohk.io" 
  Connection Attempt Start, destination 18.133.40.48:3001
[cardano.node.DnsSubscription:Notice:73] ... Domain: "relays-new.cardano-mainnet.iohk.io" 
  Connection Attempt End, destination 18.223.202.44:3001 outcome: ConnectSuccessLast
[cardano.node.ErrorPolicy:Notice:56] ... IP 18.157.126.47:3001 ErrorPolicySuspendConsumer 
  (Just (ConnectionExceptionTrace (SubscriberError {seType = SubscriberParallelConnectionCancelled, 
  seMessage = "Parallel connection cancelled", seStack = []}))) 1s
[cardano.node.ErrorPolicy:Notice:56] ... IP 18.133.40.48:3001 ErrorPolicySuspendConsumer 
  (Just (ConnectionExceptionTrace (SubscriberError {seType = SubscriberParallelConnectionCancelled, 
  seMessage = "Parallel connection cancelled", seStack = []}))) 1s

4. Synchronization

Finally the node starts fetching new blocks and applying the ledger rules to each of them to create its copy of the blockchain. 

[cardano.node.ChainDB:Notice:37] ... Chain extended, new tip: 681662266157c173b580ce3c0d5aadfb4bf09c40bef39bdc3e12fc849e4cd61e at slot 39415881
[cardano.node.ChainDB:Notice:37] ... Chain extended, new tip: cef28288211af3a2549a1fdf89c233798ee6fc94ad497b3f9c96f694cb55c2e2 at slot 39415962

 

Use your node logs to diagnose problems:

Connecting to network is, perhaps, one of the most frequent problems you can encounter.

Screen_Shot_2021-09-07_at_3.17.29.png

There can be several reasons for this, below we explore the most common:

 

1. The node is still calculating the LedgerState, and it has not reached the point where it can open the blockchain databases. When this is the case, typically the last line you see in the log file from a running node is as follows:

[cardano.node.ip-producers:Notice:5] [Date Time] IPSubscriptionTarget {ispIps = [], ispValency = 0}

It can remain like this for a very long time, even a few hours, depending on the DB state, CPU and RAM of your system. 

Solution:

  • Make sure your machine meets the System Requirements
  • Optimize your machine performance (see: 'Connecting to network' message)
  • Wait for the process to complete. 
  • To prevent this, always close Daedalus correctly (i.e. Do not close your laptop with Daedalus opened.)

2. Verifying the blockchain when starting Daedalus you might see: "Verifying the blockchain (%complete)". It can start from 0% or from a more recent point in time, i.e. 80%.

Screen_Shot_2021-09-07_at_0.41.48.png


The node.log says:

[iMac:cardano.node.ChainDB:Info:31] [Date Time] Replayed block: slot SlotNo 32767193 of At (SlotNo 39074363)
[iMac:cardano.node.ChainDB:Info:31] [Date Time] block replay progress (%) = 83.9

 

Other times you might see your node "stuck" in

[cardano.node.ChainDB:Info:31] ... Opened lgr db

When the Ledger state snapshot stored in your disk is older than the immutable tip, the node has to reapply the blocks after the snapshot to obtain the ledger state at the immutable tip.

If there is no valid snapshot to try, the node has to reapply all blocks starting from genesis to obtain the ledger state at the immutable tip.

Taking long time frequently means some data corruption has occurred the last time you closed Daedalus. i.e. it was not closed cleanly, was shutdown in the middle of a blockchain verification process, etc. 

 

Solution:

  • Make sure your machine meets the System Requirements
  • Optimize your machine performance (see: 'Connecting to network' message)
  • No workaround, the process needs to complete
  • It can take several hours, so it is a good idea to let Daedalus run overnight if it started from 0%
  • Make sure to disable sleep mode so that the process does not get interrupted
  • Do not restart Daedalus or the node during the process
  • Synchronizing your wallet frequently can prevent this

3. Failed to start all required subscriptions.  When the node fails to start the required subscriptions, and the logs do not show any other error message, it is likely that there is a local firewall rule blocking cardano-node. This rule might not affect any other services or applications. 

node.log

[cardano.node.DnsSubscription...] Domain: "relays-new.cardano-mainnet.iohk.io"
Failed to start all required subscription
[cardano.node.DnsSubscription...] Domain: "relays-new.cardano-mainnet.iohk.io"
Failed to start all required subscription
[cardano.node.DnsSubscription...] Domain: "relays-new.cardano-mainnet.iohk.io"
Failed to start all required subscription
[cardano.node.DnsSubscription...] Domain: "relays-new.cardano-mainnet.iohk.io"
Failed to start all required subscription
[cardano.node.DnsSubscription...] Domain: "relays-new.cardano-mainnet.iohk.io"
Failed to start all required subscription
[cardano.node.DnsSubscription...] Domain: "relays-new.cardano-mainnet.iohk.io"
Failed to start all required subscription
[cardano.node.DnsSubscription...] Domain: "relays-new.cardano-mainnet.iohk.io"
Failed to start all required subscription
[cardano.node.DnsSubscription...] Domain: "relays-new.cardano-mainnet.iohk.io"
Failed to start all required subscription

 

Solution:

Check your firewall rules to ensure cardano-node outgoing connections are allowed.

  • Windows: Verify the outgoing rules for the ACTIVE network profile. If needed, create an OUTBOUND rule for cardano-node and cardano-wallet 
  • macOS: macOS firewall allows all outgoing connections by default. If you are using macOS firewall, it should not be causing any problems. If you are using an alternative firewall, allow outgoing connections for Daedalus, cardano-node, and cardano-wallet. 
  • Linux: Allow outgoing connections. 

4. NTP Servers unreachable + Failed to start all required subscriptions. Similar to the one above, but this time Daedalus also shows a red screen saying that NTP servers are unreachable. It is likely that there is a local firewall rule blocking cardano-node and cardano-wallet or a failure in the internet connection. 

node.log

[cardano.node.DnsSubscription...] Domain: "relays-new.cardano-mainnet.iohk.io"
Failed to start all required subscription
[cardano.node.DnsSubscription...] Domain: "relays-new.cardano-mainnet.iohk.io"
Failed to start all required subscription
[cardano.node.DnsSubscription...] Domain: "relays-new.cardano-mainnet.iohk.io"
Failed to start all required subscription

Screen_Shot_2021-09-09_at_10.25.46.png

Solution:

  • Check your internet connection, it might be off.
  • Check your firewall rules to ensure cardano-node AND cardano-wallet outgoing connections are allowed. Note that there might be a broader rule in your firewall that blocks them both. 
  • Less frequently, an ISP might have some restrictive rules in place. Try using a different ISP. 

5. Connection timed out (WSAETIMEDOUT) This error message usually means that a network firewall (not your local firewall) is blocking the connection. Usually happens when you are connecting at a corporate network (company or office network) or a public network (hotel, airports, etc.)

node.log

[cardano.node.ErrorPolicy:Notice:44]... IP 204.236.187.238:3001 ErrorPolicySuspendConsumer 
(Just (ConnectionExceptionTrace Network.Socket.connect: <socket: 900>: failed 
(Connection timed out (WSAETIMEDOUT)))) 20s

Solution:

  • Use a different network, or
  • Talk to the network administrator.

6. Exceeded time limit.  The servers failed to reply on time over the chainsync protocol. The server has 10 seconds in which it must either reply with a roll forward/backward message, or tell the client to wait longer.

node.log

[cardano.node.IpSubscription:Error:77882]IPs: ...  Application Exception: ...
ExceededTimeLimit (ChainSync (Header (HardForkBlock (': * ByronBlock (': * ...

Solution:

  • Wait a few minutes, the node will try to connect to a different relay node (peer) 

 

See other Known Issues - all Daedalus wallet editions 

 

For assistance, please submit a ticket to the IOHK support team from the Help menu in Daedalus, or by using the submit a request form. For timely resolution, please be sure to send your logs with your request. Please see also How to download log files

Related articles