Mercurial > hg > pnewss

annotate fetchnews.php @ 12:7917bd536187 draft

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
author Ivo Smits <Ivo@UCIS.nl>
date Wed, 12 Jun 2013 22:22:07 +0200
parents e0807e0b1a67
children cccd73f72bf6
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3
0dcdb73cbcbf Increased article body length to 16MB in database scheme, cleaned up the code, added a script to forcefully reload an individual article
Ivo Smits <Ivo@UCIS.nl>
parents: 2
diff changeset
1 #!/usr/bin/php
0
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
2 <?php
8
005339a1b2ce Added copyright notice
Ivo Smits <Ivo@UCIS.nl>
parents: 7
diff changeset
3 /* Copyright 2010 Ivo Smits <Ivo@UCIS.nl>. All rights reserved.
005339a1b2ce Added copyright notice
Ivo Smits <Ivo@UCIS.nl>
parents: 7
diff changeset
4 Redistribution and use in source and binary forms, with or without modification, are
005339a1b2ce Added copyright notice
Ivo Smits <Ivo@UCIS.nl>
parents: 7
diff changeset
5 permitted provided that the following conditions are met:
005339a1b2ce Added copyright notice
Ivo Smits <Ivo@UCIS.nl>
parents: 7
diff changeset
6
005339a1b2ce Added copyright notice
Ivo Smits <Ivo@UCIS.nl>
parents: 7
diff changeset
7 1. Redistributions of source code must retain the above copyright notice, this list of
005339a1b2ce Added copyright notice
Ivo Smits <Ivo@UCIS.nl>
parents: 7
diff changeset
8 conditions and the following disclaimer.
005339a1b2ce Added copyright notice
Ivo Smits <Ivo@UCIS.nl>
parents: 7
diff changeset
9
005339a1b2ce Added copyright notice
Ivo Smits <Ivo@UCIS.nl>
parents: 7
diff changeset
10 2. Redistributions in binary form must reproduce the above copyright notice, this list
005339a1b2ce Added copyright notice
Ivo Smits <Ivo@UCIS.nl>
parents: 7
diff changeset
11 of conditions and the following disclaimer in the documentation and/or other materials
005339a1b2ce Added copyright notice
Ivo Smits <Ivo@UCIS.nl>
parents: 7
diff changeset
12 provided with the distribution.
005339a1b2ce Added copyright notice
Ivo Smits <Ivo@UCIS.nl>
parents: 7
diff changeset
13
005339a1b2ce Added copyright notice
Ivo Smits <Ivo@UCIS.nl>
parents: 7
diff changeset
14 THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED
005339a1b2ce Added copyright notice
Ivo Smits <Ivo@UCIS.nl>
parents: 7
diff changeset
15 WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
005339a1b2ce Added copyright notice
Ivo Smits <Ivo@UCIS.nl>
parents: 7
diff changeset
16 FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR
005339a1b2ce Added copyright notice
Ivo Smits <Ivo@UCIS.nl>
parents: 7
diff changeset
17 CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
005339a1b2ce Added copyright notice
Ivo Smits <Ivo@UCIS.nl>
parents: 7
diff changeset
18 CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
005339a1b2ce Added copyright notice
Ivo Smits <Ivo@UCIS.nl>
parents: 7
diff changeset
19 SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
005339a1b2ce Added copyright notice
Ivo Smits <Ivo@UCIS.nl>
parents: 7
diff changeset
20 ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
005339a1b2ce Added copyright notice
Ivo Smits <Ivo@UCIS.nl>
parents: 7
diff changeset
21 NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
005339a1b2ce Added copyright notice
Ivo Smits <Ivo@UCIS.nl>
parents: 7
diff changeset
22 ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
005339a1b2ce Added copyright notice
Ivo Smits <Ivo@UCIS.nl>
parents: 7
diff changeset
23
005339a1b2ce Added copyright notice
Ivo Smits <Ivo@UCIS.nl>
parents: 7
diff changeset
24 The views and conclusions contained in the software and documentation are those of the
005339a1b2ce Added copyright notice
Ivo Smits <Ivo@UCIS.nl>
parents: 7
diff changeset
25 authors and should not be interpreted as representing official policies, either expressed
005339a1b2ce Added copyright notice
Ivo Smits <Ivo@UCIS.nl>
parents: 7
diff changeset
26 or implied, of Ivo Smits.*/
005339a1b2ce Added copyright notice
Ivo Smits <Ivo@UCIS.nl>
parents: 7
diff changeset
27
2
40e545510a57 Added support for POSTing to the server, added readme and todo, added support for upstream synchronization using POST
Ivo Smits <Ivo@UCIS.nl>
parents: 1
diff changeset
28 chdir(__DIR__);
3
0dcdb73cbcbf Increased article body length to 16MB in database scheme, cleaned up the code, added a script to forcefully reload an individual article
Ivo Smits <Ivo@UCIS.nl>
parents: 2
diff changeset
29 require_once './common.php';
0
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
30
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
31 foreach ($db->evalAllAssoc('SELECT * FROM `peers`') as $peer) {
12
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
32 if (!$peer['enabled']) continue;
0
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
33 $socket = stream_socket_client($peer['address']);
11
e0807e0b1a67 Added common.php, updated server synchronization to continue after a connection failure
Ivo Smits <Ivo@UCIS.nl>
parents: 9
diff changeset
34 if ($socket === FALSE) {
e0807e0b1a67 Added common.php, updated server synchronization to continue after a connection failure
Ivo Smits <Ivo@UCIS.nl>
parents: 9
diff changeset
35 print("Could not connect to peer $peer[address]\n");
e0807e0b1a67 Added common.php, updated server synchronization to continue after a connection failure
Ivo Smits <Ivo@UCIS.nl>
parents: 9
diff changeset
36 continue;
e0807e0b1a67 Added common.php, updated server synchronization to continue after a connection failure
Ivo Smits <Ivo@UCIS.nl>
parents: 9
diff changeset
37 }
0
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
38 $line = nntp_readline($socket);
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
39 $code = strtok($line, " \t");
2
40e545510a57 Added support for POSTing to the server, added readme and todo, added support for upstream synchronization using POST
Ivo Smits <Ivo@UCIS.nl>
parents: 1
diff changeset
40 if ($code == 200) {
40e545510a57 Added support for POSTing to the server, added readme and todo, added support for upstream synchronization using POST
Ivo Smits <Ivo@UCIS.nl>
parents: 1
diff changeset
41 } else if ($code == 201) {
40e545510a57 Added support for POSTing to the server, added readme and todo, added support for upstream synchronization using POST
Ivo Smits <Ivo@UCIS.nl>
parents: 1
diff changeset
42 $peer['post'] = 0;
40e545510a57 Added support for POSTing to the server, added readme and todo, added support for upstream synchronization using POST
Ivo Smits <Ivo@UCIS.nl>
parents: 1
diff changeset
43 } else die("Error code $code from $peer[address]\n");
12
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
44 while ($peer['post'] == 2) {
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
45 if ($peer['lastposted'] === NULL) {
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
46 $articles = $db->evalAllAssoc('SELECT * FROM `messages` ORDER BY `id` ASC LIMIT 10');
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
47 } else {
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
48 $articles = $db->evalAllAssoc('SELECT * FROM `messages` WHERE `id` > ? ORDER BY `id` ASC LIMIT 10', $peer['lastposted']);
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
49 }
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
50 if (!count($articles)) break;
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
51 foreach ($articles as $article) {
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
52 nntp_writeline($socket, 'IHAVE <'.$article['messageid'].'>');
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
53 $line = nntp_readline($socket);
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
54 $code = strtok($line, " \t");
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
55 if ($code == 435) { //Duplicate
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
56 } else if ($code == 335) { //Please send
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
57 foreach (explode("\r\n", $article['header']) as $line) nntp_writeline_data($socket, $line);
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
58 nntp_writeline($socket, '');
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
59 foreach (explode("\r\n", $article['body']) as $line) nntp_writeline_data($socket, $line);
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
60 nntp_writeline($socket, '.');
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
61 $line = nntp_readline($socket);
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
62 $code = strtok($line, " \t");
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
63 if ($code != 240 && $code != 235) print("Article $article[messageid] was not accepted ($code)\n");
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
64 if ($article['id'] > $peer['lastposted']) $peer['lastposted'] = $article['id'];
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
65 } else {
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
66 print("IHAVE rejected by remote server, falling back to POST\n");
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
67 $peer['post'] = 1;
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
68 break;
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
69 }
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
70 if ($article['id'] > $peer['lastposted']) $peer['lastposted'] = $article['id'];
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
71 }
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
72 $db->update('UPDATE `peers` SET `lastposted` = ? WHERE `id` = ?', array($peer['lastposted'], $peer['id']));
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
73 }
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
74 nntp_writeline($socket, 'MODE READER');
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
75 $line = nntp_readline($socket);
0
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
76 foreach ($db->evalAllAssoc('SELECT * FROM `peergroups` WHERE `peer` = ?', $peer['id']) as $peergroup) {
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
77 $group = $db->evalRowAssoc('SELECT * FROM `groups` WHERE `id` = ?', $peergroup['group']);
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
78 nntp_writeline($socket, 'GROUP '.$group['name']);
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
79 $line = nntp_readline($socket);
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
80 $code = strtok($line, " \t");
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
81 if ($code != 211) die("Error code $code from $peer[address]\n");
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
82 strtok(" \t"); //number of articles
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
83 $low = strtok(" \t");
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
84 $high = strtok(" \t");
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
85 strtok(" \t"); //group name
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
86 if ($low != $peergroup['low'] || $high != $peergroup['high'] || $peergroup['low'] === NULL || $peergroup['high'] === NULL) {
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
87 for ($i = $low; $i <= $high; $i++) {
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
88 if ($i >= $peergroup['low'] && $i <= $peergroup['high'] && $peergroup['low'] !== NULL && $peergroup['high'] !== NULL) continue;
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
89 nntp_writeline($socket, 'STAT '.$i);
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
90 $line = nntp_readline($socket);
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
91 $code = strtok($line, " \t");
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
92 if ($code == 423) {
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
93 print("Gap in article numbering at $i\n");
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
94 continue;
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
95 }
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
96 if ($code != 223) die("Error code $code from $peer[address]\n");
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
97 strtok(" \t"); //article number
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
98 $messageid = strtok(" \t");
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
99 if ($messageid[0] != '<' || $messageid[strlen($messageid)-1] != '>') die("Malformed message ID $messageid\n");
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
100 $messageid = substr($messageid, 1, -1);
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
101 $message = $db->evalRowAssoc('SELECT * FROM `messages` WHERE `messageid` = ?', $messageid);
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
102 if ($message) {
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
103 $groupmessage = $db->evalRowAssoc('SELECT * FROM `groupmessages` WHERE `group` = ? AND `message` = ?', array($group['id'], $message['id']));
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
104 if (!$groupmessage) {
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
105 $db->insert('INSERT INTO `groupmessages` (`group`, `message`) VALUES (?, ?)', array($group['id'], $message['id']));
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
106 }
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
107 } else {
3
0dcdb73cbcbf Increased article body length to 16MB in database scheme, cleaned up the code, added a script to forcefully reload an individual article
Ivo Smits <Ivo@UCIS.nl>
parents: 2
diff changeset
108 nntp_writeline($socket, 'ARTICLE '.$i);
0
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
109 $line = nntp_readline($socket);
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
110 $code = strtok($line, " \t");
3
0dcdb73cbcbf Increased article body length to 16MB in database scheme, cleaned up the code, added a script to forcefully reload an individual article
Ivo Smits <Ivo@UCIS.nl>
parents: 2
diff changeset
111 if ($code != 220) die("Error code $code from $peer[address]\n");
0
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
112 strtok(" \t"); //article number
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
113 $lines = nntp_readlines($socket);
3
0dcdb73cbcbf Increased article body length to 16MB in database scheme, cleaned up the code, added a script to forcefully reload an individual article
Ivo Smits <Ivo@UCIS.nl>
parents: 2
diff changeset
114 try {
0dcdb73cbcbf Increased article body length to 16MB in database scheme, cleaned up the code, added a script to forcefully reload an individual article
Ivo Smits <Ivo@UCIS.nl>
parents: 2
diff changeset
115 nntp_article_store($lines);
0dcdb73cbcbf Increased article body length to 16MB in database scheme, cleaned up the code, added a script to forcefully reload an individual article
Ivo Smits <Ivo@UCIS.nl>
parents: 2
diff changeset
116 } catch (Exception $ex) {
0dcdb73cbcbf Increased article body length to 16MB in database scheme, cleaned up the code, added a script to forcefully reload an individual article
Ivo Smits <Ivo@UCIS.nl>
parents: 2
diff changeset
117 writelog($ex->getMessage());
0dcdb73cbcbf Increased article body length to 16MB in database scheme, cleaned up the code, added a script to forcefully reload an individual article
Ivo Smits <Ivo@UCIS.nl>
parents: 2
diff changeset
118 }
0
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
119 }
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
120 }
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
121 $db->update('UPDATE `peergroups` SET `low` = ?, `high` = ? WHERE `peer` = ? AND `group` = ?', array($low, $high, $peergroup['peer'], $peergroup['group']));
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
122 }
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
123 }
12
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
124 while ($peer['post'] == 1) {
2
40e545510a57 Added support for POSTing to the server, added readme and todo, added support for upstream synchronization using POST
Ivo Smits <Ivo@UCIS.nl>
parents: 1
diff changeset
125 if ($peer['lastposted'] === NULL) {
12
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
126 $articles = $db->evalAllAssoc('SELECT * FROM `messages` ORDER BY `id` ASC LIMIT 5');
2
40e545510a57 Added support for POSTing to the server, added readme and todo, added support for upstream synchronization using POST
Ivo Smits <Ivo@UCIS.nl>
parents: 1
diff changeset
127 } else {
12
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
128 $articles = $db->evalAllAssoc('SELECT * FROM `messages` WHERE `id` > ? ORDER BY `id` ASC LIMIT 5', $peer['lastposted']);
2
40e545510a57 Added support for POSTing to the server, added readme and todo, added support for upstream synchronization using POST
Ivo Smits <Ivo@UCIS.nl>
parents: 1
diff changeset
129 }
40e545510a57 Added support for POSTing to the server, added readme and todo, added support for upstream synchronization using POST
Ivo Smits <Ivo@UCIS.nl>
parents: 1
diff changeset
130 if (!count($articles)) break;
40e545510a57 Added support for POSTing to the server, added readme and todo, added support for upstream synchronization using POST
Ivo Smits <Ivo@UCIS.nl>
parents: 1
diff changeset
131 foreach ($articles as $article) {
12
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
132 nntp_writeline($socket, 'STAT <'.$article['messageid'].'>');
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
133 $line = nntp_readline($socket);
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
134 $code = strtok($line, " \t");
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
135 if ($code == 501) { //Argument error
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
136 print("STAT rejected by remote server, skipping message\n");
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
137 } elseif ($code == 223) { //Exists
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
138 } elseif ($code == 430) { //Not found
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
139 nntp_writeline($socket, 'POST');
9
ae0c67d72670 Bugfix in server POST code, add support for client side IHAVE and fall back to STAT-before-POST
Ivo Smits <Ivo@UCIS.nl>
parents: 8
diff changeset
140 $line = nntp_readline($socket);
ae0c67d72670 Bugfix in server POST code, add support for client side IHAVE and fall back to STAT-before-POST
Ivo Smits <Ivo@UCIS.nl>
parents: 8
diff changeset
141 $code = strtok($line, " \t");
12
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
142 if ($code != 340) die("Error code $code from $peer[address]\n");
9
ae0c67d72670 Bugfix in server POST code, add support for client side IHAVE and fall back to STAT-before-POST
Ivo Smits <Ivo@UCIS.nl>
parents: 8
diff changeset
143 foreach (explode("\r\n", $article['header']) as $line) nntp_writeline_data($socket, $line);
ae0c67d72670 Bugfix in server POST code, add support for client side IHAVE and fall back to STAT-before-POST
Ivo Smits <Ivo@UCIS.nl>
parents: 8
diff changeset
144 nntp_writeline($socket, '');
ae0c67d72670 Bugfix in server POST code, add support for client side IHAVE and fall back to STAT-before-POST
Ivo Smits <Ivo@UCIS.nl>
parents: 8
diff changeset
145 foreach (explode("\r\n", $article['body']) as $line) nntp_writeline_data($socket, $line);
ae0c67d72670 Bugfix in server POST code, add support for client side IHAVE and fall back to STAT-before-POST
Ivo Smits <Ivo@UCIS.nl>
parents: 8
diff changeset
146 nntp_writeline($socket, '.');
ae0c67d72670 Bugfix in server POST code, add support for client side IHAVE and fall back to STAT-before-POST
Ivo Smits <Ivo@UCIS.nl>
parents: 8
diff changeset
147 $line = nntp_readline($socket);
ae0c67d72670 Bugfix in server POST code, add support for client side IHAVE and fall back to STAT-before-POST
Ivo Smits <Ivo@UCIS.nl>
parents: 8
diff changeset
148 $code = strtok($line, " \t");
ae0c67d72670 Bugfix in server POST code, add support for client side IHAVE and fall back to STAT-before-POST
Ivo Smits <Ivo@UCIS.nl>
parents: 8
diff changeset
149 if ($code != 240 && $code != 235) print("Article $article[messageid] was not accepted ($code)\n");
12
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
150 } else {
7917bd536187 Added hook for new articles, detect send/write failures, fixed handling of multiline headers, add Date header if it doesn't exist, add option to disable peers, fixes for synchronization with INN, added streaming mode support, small fixes
Ivo Smits <Ivo@UCIS.nl>
parents: 11
diff changeset
151 die("Error code $code from $peer[address]\n");
9
ae0c67d72670 Bugfix in server POST code, add support for client side IHAVE and fall back to STAT-before-POST
Ivo Smits <Ivo@UCIS.nl>
parents: 8
diff changeset
152 }
2
40e545510a57 Added support for POSTing to the server, added readme and todo, added support for upstream synchronization using POST
Ivo Smits <Ivo@UCIS.nl>
parents: 1
diff changeset
153 if ($article['id'] > $peer['lastposted']) $peer['lastposted'] = $article['id'];
40e545510a57 Added support for POSTing to the server, added readme and todo, added support for upstream synchronization using POST
Ivo Smits <Ivo@UCIS.nl>
parents: 1
diff changeset
154 }
40e545510a57 Added support for POSTing to the server, added readme and todo, added support for upstream synchronization using POST
Ivo Smits <Ivo@UCIS.nl>
parents: 1
diff changeset
155 $db->update('UPDATE `peers` SET `lastposted` = ? WHERE `id` = ?', array($peer['lastposted'], $peer['id']));
40e545510a57 Added support for POSTing to the server, added readme and todo, added support for upstream synchronization using POST
Ivo Smits <Ivo@UCIS.nl>
parents: 1
diff changeset
156 }
0
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
157 nntp_writeline($socket, 'QUIT');
9
ae0c67d72670 Bugfix in server POST code, add support for client side IHAVE and fall back to STAT-before-POST
Ivo Smits <Ivo@UCIS.nl>
parents: 8
diff changeset
158 nntp_readline($socket);
0
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
159 fclose($socket);
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
160 }
d7ab68b71c74 Initial commit
Ivo Smits <Ivo@UCIS.nl>
parents:
diff changeset
161
3
0dcdb73cbcbf Increased article body length to 16MB in database scheme, cleaned up the code, added a script to forcefully reload an individual article
Ivo Smits <Ivo@UCIS.nl>
parents: 2
diff changeset
162 function writelog($line) {
0dcdb73cbcbf Increased article body length to 16MB in database scheme, cleaned up the code, added a script to forcefully reload an individual article
Ivo Smits <Ivo@UCIS.nl>
parents: 2
diff changeset
163 print($line."\n");
1
61fac319ca3e Update Path and Xref header fields while fetching messages, added database schema export
Ivo Smits <Ivo@UCIS.nl>
parents: 0
diff changeset
164 }