Spool Over Quota. Router limit exceeded

sulfred

I am using a guaranteed message queue. This is sit under the default VPN. The `Configured Quota` is 5,000 MB for the queue. The `Configured Quote` is 1,500 MB for the VPN.

I cannot push to the queue when the message size hit 1,500 MB. I updated the `Configured Quote` for the VPN to 10,000 MB but I still cannot push message to my queue and I hit the same issue and error message. I would like to ask how can I solve this issue?

Here is my setting on the web portal:

Thanks

TomF

Hi @sulfred, there are 3 spool quotas to consider:

1) the individual queue spool quota;

2) the message-vpn spool quota;

3) the system spool quota.

Exceeding any of these will mean your message is rejected. You seem to have received a "Router limit exceeded" error which points to the system spool quota being exceeded. To check, open PubSub+ manager. Before you go in to the Message-VPN screen, pick "Guaranteed Messaging" from the left hand menu, then pick settings from the tabs at the top. This will get you to the system level spool settings, and you can see the system level quota there.

TomF

Hi @sulfred, there are 3 spool quotas to consider:

1) the individual queue spool quota;

2) the message-vpn spool quota;

3) the system spool quota.

Exceeding any of these will mean your message is rejected. You seem to have received a "Router limit exceeded" error which points to the system spool quota being exceeded. To check, open PubSub+ manager. Before you go in to the Message-VPN screen, pick "Guaranteed Messaging" from the left hand menu, then pick settings from the tabs at the top. This will get you to the system level spool settings, and you can see the system level quota there.

sulfred

Thanks Tom. I saw that there are 3 level of configuration. The issue is that after I updated the config, I still cannot push data into the message queue. It seems to me that the config is not updated.

At the end, I manage that by removing the message from the queue first.

So, no further issue now. Thanks

Aaron

Wait, that's weird. The queue was still blocked even though you increased the quota at both the VPN level and also the System level?? That shouldn't be. You should be able to resume publishing to the queue after upping both of those other limits.

Can you confirm please?

sulfred

Yes, as the picture I included in my question. I screenshot all 3 configs and I cannot push message to the queue at the moment.

Thanks

Aaron

@sulfred sorry for the delay! I just tried reproducing this and it worked as I expected... after resizing the VPN and broker limits above 1500MB, I was able to continue publishing.

I'm wondering if something else went wrong, perhaps with the disk resources inside the Docker container... like it could free up / allocate any more space. May I ask how you instantiated the Docker container? Is there any limit on the volume sizes allocated?

sulfred

Hi @Aaron ,

Thanks for your time keep investigate this issue.

Here is the docker compose file I am using.

```

# https://github.com/SolaceLabs/solace-single-docker-compose/blob/master/template/PubSubStandard_singleNode.yml

version: '3.3'

services:

 solace:

  container_name: solaceMessageBorker

  image: solace/solace-pubsub-standard:9.12.1.17

  ulimits:

   core: 1

   nofile:

    soft: 2448

    hard: 6592

  deploy:

   restart_policy:

    condition: on-failure

    max_attempts: 1

  shm_size: 2g

  networks:

   - solace_net

  environment:

   username_admin_globalaccesslevel: "admin"

   username_admin_password: "admin"

  ports:

   # Web transport

   - "${SOLACE_WEB_TRANSPORT:-20060}:8080"

   # Solace APIs to connect to the message broker on this port

   - "${SOLACE_API_MSG_BROKER:-20061}:55555"

   # Solace APIs to connect to the message broker on this port secured

   - "${SOLACE_API_MSG_BROKER_SECURED:-20066}:55443"

   # Solace APIs to connect to the message broker on this port compressed

   - "${SOLACE_API_MSG_BROKER_COMPRESSED:-20067}:55003"

   # JavaScript sample applications pass Web Messaging traffic through

   # - "8008:8008"

   # MQTT connectitvity

   - "${SOLACE_MQTT_1:-20062}:1883"

   - "${SOLACE_MQTT_2:-20063}:8000"

   # AMQP 1.0 application using Apache QPID APIs

   - "${SOLACE_AMQP:-20064}:5672"

   # Use REST to send messaging and event data with Solace's RESTful API port

   - "${SOLACE_REST:-20065}:9000"

   # SSH Command Line Interface for advanced configuration

   - "${SOLACE_SSH:-2222}:2222"

networks:

 solace_net:

  driver: bridge

```

TomF

@sulfred , it looks like you're using the default Docker configuration. That uses a local disk, which may fill up and prevent reception of messages. Could you post the exact error your publisher sees? I think the exception you see when the disk is full is different to the quota being exceeded.

Aaron

Exactly! Would be good to know the exact issue. If you have access to CLI, you could run the command show log event find quota or show log event find exceed and see what you get back..? That searches the broker's log file for any alert like that.

sulfred

Thanks for keep helping me to troubleshoot this issue.

I am trying to reproduce the issue but failed. I think just let go for now. 😆

If I have any updates in the future, I will update again in this thread.