Discussion:
Interacting with cnfs
(too old to reply)
Nigel Reed
2024-07-05 07:36:21 UTC
Permalink
Hi all,

I was thinking about a report I did for my older server, which used the
tradspool storage method, and that was to count how many articles are
in a group and list how many new articles have been added since last
run.

I can't see a way to make this happen using cnfs without pulling the
storage token using a similar method as 6.4 in the FAQ and then
manually parsing the newsgroups line from each article.

Do we have any better tools for interacting with cnfs...or if not,
should we?

A similar thought that when looking for text, I could grep through a
group or set of groups. Not sure this would even be possible with cnfs
either?

open to thoughts...suggestions...discussion?
--
End Of The Line BBS - Plano, TX
telnet endofthelinebbs.com 23
Billy G. (go-while)
2024-07-05 09:33:58 UTC
Permalink
Post by Nigel Reed
I was thinking about a report I did for my older server, which used the
tradspool storage method, and that was to count how many articles are
in a group and list how many new articles have been added since last
run.
Why not use the numbers from active file?
Post by Nigel Reed
A similar thought that when looking for text, I could grep through a
group or set of groups. Not sure this would even be possible with cnfs
either?
That's sadly not possible with cyclic buffers :/
--
.......
Billy G. (go-while)
Nigel Reed
2024-07-06 02:35:20 UTC
Permalink
On Fri, 5 Jul 2024 11:33:58 +0200
Post by Billy G. (go-while)
Post by Nigel Reed
I was thinking about a report I did for my older server, which used
the tradspool storage method, and that was to count how many
articles are in a group and list how many new articles have been
added since last run.
Why not use the numbers from active file?
Because it doesn't take into account deleted articles.
Post by Billy G. (go-while)
Post by Nigel Reed
A similar thought that when looking for text, I could grep through
a group or set of groups. Not sure this would even be possible
with cnfs either?
That's sadly not possible with cyclic buffers :/
--
End Of The Line BBS - Plano, TX
telnet endofthelinebbs.com 23
Ray Banana
2024-07-05 11:09:52 UTC
Permalink
Post by Nigel Reed
I was thinking about a report I did for my older server, which used the
tradspool storage method, and that was to count how many articles are
in a group and list how many new articles have been added since last
run.
Are you familiar with the cnfsstat utility?
It provides part of the information you are looking for:

Class NLS for groups matching "at.*,aus.*,be.*,cz.*ch.*,dk.*,es.*,fr.*,hr.*,it.*,nl.*,no.*,nz.*,pl.*,pt.*,se.*,uk.*,yu.*"
Buffer LNLS00, size: 9.98 GBytes, position: 4.29 GBytes 1.43 cycles
Newest: 2024-07-05 13:03:12, 0 days, 0:00:56 ago
Oldest: 2023-04-02 2:43:08, 460 days, 10:21:00 ago
Buffer LNLS01, size: 40.0 GBytes, position: 1.25 MBytes 1.00 cycles
Newest: 2023-05-01 22:49:34, 430 days, 14:14:34 ago
Oldest: 2023-04-02 22:08:40, 459 days, 14:55:28 ago
Post by Nigel Reed
I can't see a way to make this happen using cnfs without pulling the
storage token using a similar method as 6.4 in the FAQ and then
manually parsing the newsgroups line from each article.
cnfsstat is written in Perl and is included in the INN sources.
The code may give you an idea of how to retrieve additional information
about the articles stored in CNFS buffers.
--
Пу́тін — хуйло́
https://www.eternal-september.org
Julien ÉLIE
2024-07-14 07:17:31 UTC
Permalink
Hi Nigel,
Post by Nigel Reed
I was thinking about a report I did for my older server, which used the
tradspool storage method, and that was to count how many articles are
in a group and list how many new articles have been added since last
run.
I can't see a way to make this happen using cnfs without pulling the
storage token using a similar method as 6.4 in the FAQ and then
manually parsing the newsgroups line from each article.
You can use overview information to get the number of articles in a
newsgroup.

% getlist -R counts
news.software.nntp 17619 1 13223 y

gives you a list of newsgroups containing the information (13223
articles in news.software.nntp, between article numbers 1 and 17619).
For accurate results, see the groupexactcount setting in readers.conf.
You may want to set it to 0 for connections coming from localhost or
whichever server running your command.

How many articles have been added since the last run is not
straight-forward though... You have to somehow record the latest high
water marks for each newsgroup, and see how many articles have a greater
article number than these recorded high water marks.

Use tdx-util or ovsqlite-util for that, assuming you have a tradindexed
or ovsqlite overview.

For instance, assuming I have recorded that the latest highest article
number for news.software.nntp was 17619, I get the number of new
articles with the following command:

% ovsqlite-util -g -n news.software.nntp -a 17620- | wc -l
0


Incidentally, you may also use tdx-util or ovsqlite-util instead of
"getlist -R counts" for the counts:

% ovsqlite-util -i -n news.software.nntp
news.software.nntp 17619 1 13223 y 1720923358 0


I hope this could help.
Post by Nigel Reed
Do we have any better tools for interacting with cnfs...or if not,
should we?
Naturally, every tool is perfectible and can be enhanced. It just needs
someone to write it and share it with the community.
Post by Nigel Reed
A similar thought that when looking for text, I could grep through a
group or set of groups. Not sure this would even be possible with cnfs
either?
You have to force a text search with "grep -a".
--
Julien ÉLIE

« Pour aller plus vite, j'additionne toujours de bas en haut : je fais
du même coup l'addition et la preuve. » (Aurélien Scholl)
Nigel Reed
2024-07-15 04:44:02 UTC
Permalink
On Sun, 14 Jul 2024 09:17:31 +0200
Post by Julien ÉLIE
Hi Nigel,
Post by Nigel Reed
I was thinking about a report I did for my older server, which used
the tradspool storage method, and that was to count how many
articles are in a group and list how many new articles have been
added since last run.
I can't see a way to make this happen using cnfs without pulling the
storage token using a similar method as 6.4 in the FAQ and then
manually parsing the newsgroups line from each article.
You can use overview information to get the number of articles in a
newsgroup.
% getlist -R counts
news.software.nntp 17619 1 13223 y
gives you a list of newsgroups containing the information (13223
articles in news.software.nntp, between article numbers 1 and 17619).
For accurate results, see the groupexactcount setting in
readers.conf. You may want to set it to 0 for connections coming from
localhost or whichever server running your command.
How many articles have been added since the last run is not
straight-forward though... You have to somehow record the latest
high water marks for each newsgroup, and see how many articles have a
greater article number than these recorded high water marks.
Use tdx-util or ovsqlite-util for that, assuming you have a
tradindexed or ovsqlite overview.
For instance, assuming I have recorded that the latest highest
article number for news.software.nntp was 17619, I get the number of
% ovsqlite-util -g -n news.software.nntp -a 17620- | wc -l
0
Incidentally, you may also use tdx-util or ovsqlite-util instead of
% ovsqlite-util -i -n news.software.nntp
news.software.nntp 17619 1 13223 y 1720923358 0
I hope this could help.
All good information, thank you. this is something that I would run
nightly so I would just need to subtract the previous nights values to
get the number of new articles per group. I'll give it a try tomorrow.
Thanks.
--
End Of The Line BBS - Plano, TX
telnet endofthelinebbs.com 23
Loading...