--MimeMultipartBoundary
Content-Type: text/plain; charset=us-ascii
Hiya,
Here is the mentioned jumbo patch to 1.2beta11 that allows for
reasonably stable ASYNC support. There are also other bits and pieces
mixed in.
It adds the following features:
- Completely rewritten aiops.c that creates and manages a pool of
threads so thread creation overhead is eliminated
- Lots of extra debugging
- Lots of mods to store.c to detect and cancel outstanding ASYNC ops.
Code is not proven exhaustive and there are definately still cases
to be found where outstanding disk ops aren't cancelled properly.
- Changes to call interface to a few routines to support disk op
`tagging', so operations can be cleanly cancelled on store_abort()s.
- Implementation of swap.state files as transaction logs. Removed
objects are now noted with a negative object size. This allows
reliatively clean rebuilds from non-clean shutdowns.
- Now that the swap.state files are transaction logs, there's now no
need to validate by stat()ing. All the validation procedure does
is now just set the valid bit AFTER all the swap.state files have
been read, because by that time, only valid objects can be left.
Object still need to be marked invalid when reading the swap.state
file because there's no guarantee the file has been retaken or
deleted.
- An fstat() call is now added after every storeSwapInFileOpened() so
object sizes can be checked. Added code to storeRelease() the
object if the sizes don't match.
- #defining USE_ASYNC_IO now uses the async unlink() rather than
unlinkd()
- #defining MONOTONIC_STORE will support the creation of disk objects
clustered into directories. This GREATLY improves disk performance
(factor of 3) over old `write-over-old-object' method. If using the
MONOTONIC_STORE, the {get/put}_unusedFileno stack stuff is disabled.
This is actually a good thing and greatly reduces the risk of serving
up bad objects.
- Fixed unlink() in storeWriteCleanLogs to be real unlink() rather
than ASYNC/unlinkd unlinks. swap.state.new files were being removed
just after they were created due to delayed unlinks.
- Disabled various assertions and made these into debug warning
messages to make the code more stable until the bugs can be tracked
down.
- Added most of Michael O'Reilly's patches which included many bug
fixes. Ask him for full details.
- Moved aio_check_callbacks in comm_{poll|select}(). It was called
after the fdset had been built which was wrong because the callbacks
were changing the state of the read/write handlers prior to the
poll/select() calls.
PLEASE can these all be included. I've tried hard to put #ifdef's
around all changes to do with MONOTONIC_STORE and ASYNC_IO such that it
will still all compile cleanly. All that is needed to to #define the
appropriate value (to 1) and it'll all work. Without defining those values
you should still get clean code with all the bug fixes.
I know it'll be some work to mix this with beta12 now, but the sooner
the better.
Cheers,
Stew.
P.S. Apply these patches against base beta11 source via
patch -p1 < diff_file
in the top level directory of the beta11 source tree.
----------------------
diff -cr squid-1.2.beta11/src/access_log.c squid-1.2.slf/src/access_log.c
*** squid-1.2.beta11/src/access_log.c Tue Jan 6 08:18:12 1998
--- squid-1.2.slf/src/access_log.c Fri Jan 9 14:37:56 1998
***************
*** 1,7 ****
/*
! * $Id: access_log.c,v 1.16 1998/01/05 21:18:12 wessels Exp $
*
* DEBUG: section 46 Access Log
* AUTHOR: Duane Wessels
--- 1,7 ----
/*
! * $Id: access_log.c,v 1.2 1998/01/09 03:37:28 slf Exp slf $
*
* DEBUG: section 46 Access Log
* AUTHOR: Duane Wessels
***************
*** 269,275 ****
/* Close and reopen the log. It may have been renamed "manually"
* before HUP'ing us. */
file_close(LogfileFD);
! LogfileFD = file_open(fname, O_WRONLY | O_CREAT, NULL, NULL);
if (LogfileFD == DISK_ERROR) {
debug(46, 0) ("accessLogRotate: Cannot open logfile: %s\n", fname);
LogfileStatus = LOG_DISABLE;
--- 269,275 ----
/* Close and reopen the log. It may have been renamed "manually"
* before HUP'ing us. */
file_close(LogfileFD);
! LogfileFD = file_open(fname, O_WRONLY | O_CREAT, NULL, NULL, NULL);
if (LogfileFD == DISK_ERROR) {
debug(46, 0) ("accessLogRotate: Cannot open logfile: %s\n", fname);
LogfileStatus = LOG_DISABLE;
***************
*** 288,294 ****
{
assert(fname);
xstrncpy(LogfileName, fname, SQUID_MAXPATHLEN);
! LogfileFD = file_open(LogfileName, O_WRONLY | O_CREAT, NULL, NULL);
if (LogfileFD == DISK_ERROR) {
debug(50, 0) ("%s: %s\n", LogfileName, xstrerror());
fatal("Cannot open logfile.");
--- 288,294 ----
{
assert(fname);
xstrncpy(LogfileName, fname, SQUID_MAXPATHLEN);
! LogfileFD = file_open(LogfileName, O_WRONLY | O_CREAT, NULL, NULL, NULL);
if (LogfileFD == DISK_ERROR) {
debug(50, 0) ("%s: %s\n", LogfileName, xstrerror());
fatal("Cannot open logfile.");
diff -cr squid-1.2.beta11/src/aiops.c squid-1.2.slf/src/aiops.c
*** squid-1.2.beta11/src/aiops.c Tue Jan 6 11:27:55 1998
--- squid-1.2.slf/src/aiops.c Mon Feb 2 14:39:07 1998
***************
*** 1,6 ****
/*
! * $Id: aiops.c,v 1.5 1998/01/06 00:27:55 wessels Exp $
*
* DEBUG: section 43 AIOPS
* AUTHOR: Stewart Forster <slf@connect.com.au>
--- 1,6 ----
/*
! * $Id: aiops.c,v 1.5 1998/01/20 06:29:45 slf Exp slf $
*
* DEBUG: section 43 AIOPS
* AUTHOR: Stewart Forster <slf@connect.com.au>
***************
*** 31,36 ****
--- 31,38 ----
#include "squid.h"
+ #define _REENTRANT
+
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
***************
*** 38,49 ****
#include <pthread.h>
#include <errno.h>
#include <dirent.h>
! #define MAXTHREADS 1024
- #define _THREAD_FREE 0
- #define _THREAD_DOING 1
- #define _THREAD_DONE 2
#define _AIO_OP_OPEN 0
#define _AIO_OP_READ 1
--- 40,55 ----
#include <pthread.h>
#include <errno.h>
#include <dirent.h>
+ #include <signal.h>
+
+ #define NUMTHREADS 16
+ #define RIDICULOUS_LENGTH 4096
! #define _THREAD_STARTING 0
! #define _THREAD_WAITING 1
! #define _THREAD_BUSY 2
! #define _THREAD_FAILED 3
#define _AIO_OP_OPEN 0
#define _AIO_OP_READ 1
***************
*** 53,112 ****
#define _AIO_OP_OPENDIR 5
#define _AIO_OP_STAT 6
! typedef struct aio_thread_t {
! pthread_t thread;
! int status;
! int operation;
aio_result_t *resultp;
! void *aiodp;
! struct aio_thread_t *next;
! } aio_thread_t;
!
!
! typedef struct aioopen_d {
! char *path;
! int oflag;
! mode_t mode;
! } aio_open_d;
!
!
! typedef struct aio_write_d {
! int fd;
! char *bufp;
! int bufs;
! off_t offset;
! int whence;
! } aio_write_d;
!
- typedef struct aio_read_d {
- int fd;
- char *bufp;
- int bufs;
- off_t offset;
- int whence;
- } aio_read_d;
-
-
- typedef struct aio_close_d {
- int fd;
- } aio_close_d;
-
-
- typedef struct aio_stat_d {
- char *path;
- struct stat *sb;
- } aio_stat_d;
-
-
- typedef struct aio_unlink_d {
- char *path;
- } aio_unlink_d;
- typedef struct aio_opendir_d {
- char *path;
- } aio_opendir_d;
int aio_cancel(aio_result_t *);
int aio_open(const char *, int, mode_t, aio_result_t *);
--- 59,95 ----
#define _AIO_OP_OPENDIR 5
#define _AIO_OP_STAT 6
! typedef struct aio_request_t {
! int request_type;
! int cancelled;
! char *path;
! int oflag;
! mode_t mode;
! int fd;
! char *bufferp;
! char *tmpbufp;
! int buflen;
! off_t offset;
! int whence;
! int ret;
! int err;
! struct stat *tmpstatp;
! struct stat *statp;
aio_result_t *resultp;
! struct aio_request_t *next;
! } aio_request_t;
+ typedef struct aio_thread_t {
+ pthread_t thread;
+ int status;
+ pthread_mutex_t mutex; /* Mutex for testing condition variable */
+ pthread_cond_t cond; /* Condition variable */
+ struct aio_request_t *req;
+ struct aio_request_t *donereq;
+ struct aio_thread_t *next;
+ } aio_thread_t;
int aio_cancel(aio_result_t *);
int aio_open(const char *, int, mode_t, aio_result_t *);
***************
*** 118,553 ****
aio_result_t *aio_poll_done();
static void aio_init(void);
- static aio_thread_t *aio_alloc_thread(aio_result_t *);
static void aio_free_thread(aio_thread_t *);
static void aio_cleanup_and_free(aio_thread_t *);
! static void *aio_thread_open(void *);
! static void *aio_thread_read(void *);
! static void *aio_thread_write(void *);
! static void *aio_thread_close(void *);
! static void *aio_thread_stat(void *);
! static void *aio_thread_unlink(void *);
#if 0
static void *aio_thread_opendir(void *);
#endif
! static void aio_debug(aio_thread_t *);
! static aio_thread_t thread[MAXTHREADS];
static int aio_initialised = 0;
! static aio_thread_t *free_threads = NULL;
! static aio_thread_t *used_threads = NULL;
! static aio_thread_t *tail_threads = NULL;
static void
aio_init(void)
{
int i;
if (aio_initialised)
return;
! for (i = 0; i < MAXTHREADS; i++) {
! thread[i].next = free_threads;
! free_threads = thread + i;
}
aio_initialised = 1;
}
! static aio_thread_t *
! aio_alloc_thread(aio_result_t * resultp)
{
! aio_thread_t *threadp;
! if (free_threads == NULL) {
! errno = EAGAIN;
! return NULL;
! }
! for (threadp = used_threads; threadp != NULL; threadp = threadp->next)
! if (threadp->resultp == resultp)
break;
! if (threadp != NULL) {
! errno = EINVAL;
! return NULL;
! }
! threadp = free_threads;
! free_threads = threadp->next;
- if (tail_threads == NULL)
- used_threads = threadp;
- else
- tail_threads->next = threadp;
- tail_threads = threadp;
! threadp->status = _THREAD_DOING;
! threadp->resultp = NULL;
! threadp->next = NULL;
! return threadp;
! }
static void
! aio_free_thread(aio_thread_t * threadp)
{
! aio_thread_t *c;
! if (threadp == NULL)
! return;
! if (used_threads == NULL)
return;
! if (used_threads == threadp) {
! if ((used_threads = threadp->next) == NULL)
! tail_threads = NULL;
} else {
! for (c = used_threads; c != NULL && c->next != threadp; c = c->next);
! if (c == NULL)
return;
! if ((c->next = threadp->next) == NULL)
! tail_threads = c;
}
! threadp->next = free_threads;
! free_threads = threadp;
! }
static void
! aio_cleanup_and_free(aio_thread_t * threadp)
{
! aio_open_d *od;
! aio_unlink_d *ud;
! aio_opendir_d *odd;
! aio_stat_d *sd;
! switch (threadp->operation) {
case _AIO_OP_OPEN:
- od = (aio_open_d *) threadp->aiodp;
- xfree(od->path);
- break;
case _AIO_OP_UNLINK:
- ud = (aio_unlink_d *) threadp->aiodp;
- xfree(ud->path);
- break;
- case _AIO_OP_STAT:
- sd = (aio_stat_d *) threadp->aiodp;
- xfree(sd->path);
- break;
case _AIO_OP_OPENDIR:
! odd = (aio_opendir_d *) threadp->aiodp;
! xfree(odd->path);
break;
default:
break;
}
! xfree(threadp->aiodp);
! aio_free_thread(threadp);
! }
int
aio_cancel(aio_result_t * resultp)
{
aio_thread_t *threadp;
int ret;
! for (threadp = used_threads; threadp != NULL; threadp = threadp->next);
! if (threadp == NULL) {
! errno = ENOENT;
! return -1;
}
! ret = pthread_cancel(threadp->thread);
! aio_cleanup_and_free(threadp);
! return ret;
! }
int
aio_open(const char *path, int oflag, mode_t mode, aio_result_t * resultp)
{
! aio_open_d *aiodp;
! aio_thread_t *threadp;
int len;
if (!aio_initialised)
aio_init();
! if ((threadp = aio_alloc_thread(resultp)) == NULL)
! return -1;
! if ((aiodp = (aio_open_d *) xmalloc(sizeof(aio_open_d))) == NULL) {
! aio_free_thread(threadp);
errno = ENOMEM;
return -1;
}
len = strlen(path) + 1;
! if ((aiodp->path = (char *) xmalloc(len)) == NULL) {
! aio_free_thread(threadp);
! xfree(aiodp);
errno = ENOMEM;
return -1;
}
! strncpy(aiodp->path, path, len);
! aiodp->oflag = oflag;
! aiodp->mode = mode;
! threadp->aiodp = aiodp;
! threadp->resultp = resultp;
! threadp->operation = _AIO_OP_OPEN;
! if (pthread_create(&(threadp->thread), NULL, aio_thread_open, threadp) <
0) {
! xfree(aiodp->path);
! xfree(aiodp);
! aio_free_thread(threadp);
! return -1;
! }
return 0;
}
! static void *
! aio_thread_open(void *ptr)
{
! aio_thread_t *threadp = (aio_thread_t *) ptr;
! aio_open_d *aiodp;
! aiodp = (aio_open_d *) threadp->aiodp;
! threadp->resultp->aio_return = open(aiodp->path, aiodp->oflag, aiodp->
mode);
! threadp->resultp->aio_errno = errno;
! threadp->status = _THREAD_DONE;
! return threadp;
}
int
aio_read(int fd, char *bufp, int bufs, off_t offset, int whence,
aio_result_t * resultp)
{
! aio_read_d *aiodp;
! aio_thread_t *threadp;
if (!aio_initialised)
aio_init();
! if ((threadp = aio_alloc_thread(resultp)) == NULL)
! return -1;
! if ((aiodp = (aio_read_d *) xmalloc(sizeof(aio_read_d))) == NULL) {
! aio_free_thread(threadp);
errno = ENOMEM;
return -1;
}
! aiodp->fd = fd;
! aiodp->bufp = bufp;
! aiodp->bufs = bufs;
! aiodp->offset = offset;
! aiodp->whence = whence;
! threadp->aiodp = aiodp;
! threadp->resultp = resultp;
! threadp->operation = _AIO_OP_READ;
! if (pthread_create(&(threadp->thread), NULL, aio_thread_read, threadp) <
0) {
! xfree(aiodp);
! aio_free_thread(threadp);
return -1;
}
return 0;
}
! static void *
! aio_thread_read(void *ptr)
{
! aio_thread_t *threadp = (aio_thread_t *) ptr;
! aio_read_d *aiodp;
! aiodp = (aio_read_d *) threadp->aiodp;
! lseek(aiodp->fd, aiodp->offset, aiodp->whence);
! threadp->resultp->aio_return = read(aiodp->fd, aiodp->bufp, aiodp->bufs);
! threadp->resultp->aio_errno = errno;
! threadp->status = _THREAD_DONE;
! return threadp;
}
int
aio_write(int fd, char *bufp, int bufs, off_t offset, int whence,
aio_result_t * resultp)
{
! aio_write_d *aiodp;
! aio_thread_t *threadp;
if (!aio_initialised)
aio_init();
! if ((threadp = aio_alloc_thread(resultp)) == NULL)
! return -1;
! if ((aiodp = (aio_write_d *) xmalloc(sizeof(aio_write_d))) == NULL) {
! aio_free_thread(threadp);
errno = ENOMEM;
return -1;
}
! aiodp->fd = fd;
! aiodp->bufp = bufp;
! aiodp->bufs = bufs;
! aiodp->offset = offset;
! aiodp->whence = whence;
! threadp->aiodp = aiodp;
! threadp->resultp = resultp;
! threadp->operation = _AIO_OP_WRITE;
! if (pthread_create(&(threadp->thread), NULL, aio_thread_write, threadp)
< 0) {
! xfree(aiodp);
! aio_free_thread(threadp);
return -1;
}
return 0;
}
! static void *
! aio_thread_write(void *ptr)
{
! aio_thread_t *threadp = (aio_thread_t *) ptr;
! aio_write_d *aiodp;
! aiodp = (aio_write_d *) threadp->aiodp;
! threadp->resultp->aio_return = write(aiodp->fd, aiodp->bufp, aiodp->
bufs);
! threadp->resultp->aio_errno = errno;
! threadp->status = _THREAD_DONE;
! return threadp;
}
int
aio_close(int fd, aio_result_t * resultp)
{
! aio_close_d *aiodp;
! aio_thread_t *threadp;
if (!aio_initialised)
aio_init();
! if ((threadp = aio_alloc_thread(resultp)) == NULL)
! return -1;
! if ((aiodp = (aio_close_d *) xmalloc(sizeof(aio_close_d))) == NULL) {
! aio_free_thread(threadp);
errno = ENOMEM;
return -1;
}
! aiodp->fd = fd;
! threadp->aiodp = aiodp;
! threadp->resultp = resultp;
! threadp->operation = _AIO_OP_CLOSE;
! if (pthread_create(&(threadp->thread), NULL, aio_thread_close, threadp)
< 0) {
! xfree(aiodp);
! aio_free_thread(threadp);
! return -1;
! }
return 0;
}
! static void *
! aio_thread_close(void *ptr)
{
! aio_thread_t *threadp = (aio_thread_t *) ptr;
! aio_close_d *aiodp;
! aiodp = (aio_close_d *) threadp->aiodp;
! threadp->resultp->aio_return = close(aiodp->fd);
! threadp->resultp->aio_errno = errno;
! threadp->status = _THREAD_DONE;
! return threadp;
}
int
aio_stat(const char *path, struct stat *sb, aio_result_t * resultp)
{
! aio_stat_d *aiodp;
! aio_thread_t *threadp;
int len;
if (!aio_initialised)
aio_init();
! if ((threadp = aio_alloc_thread(resultp)) == NULL)
! return -1;
! if ((aiodp = (aio_stat_d *) xmalloc(sizeof(aio_stat_d))) == NULL) {
! aio_free_thread(threadp);
errno = ENOMEM;
return -1;
}
len = strlen(path) + 1;
! if ((aiodp->path = (char *) xmalloc(len)) == NULL) {
! aio_free_thread(threadp);
! xfree(aiodp);
errno = ENOMEM;
return -1;
}
! strncpy(aiodp->path, path, len);
! aiodp->sb = sb;
! threadp->aiodp = aiodp;
! threadp->resultp = resultp;
! threadp->operation = _AIO_OP_STAT;
! if (pthread_create(&(threadp->thread), NULL, aio_thread_stat, threadp) <
0) {
! xfree(aiodp->path);
! xfree(aiodp);
! aio_free_thread(threadp);
return -1;
}
return 0;
}
! static void *
! aio_thread_stat(void *ptr)
{
! aio_thread_t *threadp = (aio_thread_t *) ptr;
! aio_stat_d *aiodp;
! aiodp = (aio_stat_d *) threadp->aiodp;
! threadp->resultp->aio_return = stat(aiodp->path, aiodp->sb);
! threadp->resultp->aio_errno = errno;
! threadp->status = _THREAD_DONE;
! return threadp;
}
int
aio_unlink(const char *path, aio_result_t * resultp)
{
! aio_unlink_d *aiodp;
! aio_thread_t *threadp;
int len;
if (!aio_initialised)
aio_init();
! if ((threadp = aio_alloc_thread(resultp)) == NULL)
! return -1;
! if ((aiodp = (aio_unlink_d *) xmalloc(sizeof(aio_unlink_d))) == NULL) {
! aio_free_thread(threadp);
errno = ENOMEM;
return -1;
}
len = strlen(path) + 1;
! if ((aiodp->path = (char *) xmalloc(len)) == NULL) {
! aio_free_thread(threadp);
! xfree(aiodp);
errno = ENOMEM;
return -1;
}
! strncpy(aiodp->path, path, len);
! threadp->aiodp = aiodp;
! threadp->resultp = resultp;
! threadp->operation = _AIO_OP_UNLINK;
! if (pthread_create(&(threadp->thread), NULL, aio_thread_unlink, threadp)
< 0) {
! xfree(aiodp->path);
! xfree(aiodp);
! aio_free_thread(threadp);
! return -1;
! }
return 0;
}
! static void *
! aio_thread_unlink(void *ptr)
{
! aio_thread_t *threadp = (aio_thread_t *) ptr;
! aio_unlink_d *aiodp;
! aiodp = (aio_unlink_d *) threadp->aiodp;
! threadp->resultp->aio_return = unlink(aiodp->path);
! threadp->resultp->aio_errno = errno;
! threadp->status = _THREAD_DONE;
! return threadp;
}
--- 101,695 ----
aio_result_t *aio_poll_done();
static void aio_init(void);
static void aio_free_thread(aio_thread_t *);
static void aio_cleanup_and_free(aio_thread_t *);
! static void aio_queue_request(aio_request_t *);
! static void aio_process_request_queue(void);
! static void aio_cleanup_request(aio_request_t *);
! static void *aio_thread_loop(void *);
! static void aio_thread_open(aio_thread_t *);
! static void aio_thread_read(aio_thread_t *);
! static void aio_thread_write(aio_thread_t *);
! static void aio_thread_close(aio_thread_t *);
! static void aio_thread_stat(aio_thread_t *);
! static void aio_thread_unlink(aio_thread_t *);
#if 0
static void *aio_thread_opendir(void *);
#endif
! static void aio_debug(aio_request_t *);
! static aio_thread_t thread[NUMTHREADS];
static int aio_initialised = 0;
! static int request_queue_len = 0;
! static aio_request_t *free_requests = NULL;
! static int num_free_requests = 0;
! static aio_request_t *request_queue_head = NULL;
! static aio_request_t *request_queue_tail = NULL;
! static aio_thread_t *wait_threads = NULL;
! static aio_thread_t *busy_threads_head = NULL;
! static aio_thread_t *busy_threads_tail = NULL;
! static pthread_attr_t globattr;
! static struct sched_param globsched;
static void
aio_init(void)
{
int i;
+ pthread_t self;
+ aio_thread_t *threadp;
if (aio_initialised)
return;
!
! pthread_attr_init(&globattr);
! pthread_attr_setscope(&globattr, PTHREAD_SCOPE_SYSTEM);
! globsched.sched_priority = 1;
! self = pthread_self();
! pthread_setschedparam(self, SCHED_OTHER, &globsched);
! globsched.sched_priority = 2;
! pthread_attr_setschedparam(&globattr, &globsched);
!
! /* Create threads and get them to sit in their wait loop */
!
! for (i = 0; i < NUMTHREADS; i++) {
! threadp = thread + i;
! threadp->status = _THREAD_STARTING;
! if(pthread_mutex_init(&(threadp->mutex), NULL)) {
! threadp->status = _THREAD_FAILED;
! continue;
! }
! if(pthread_cond_init(&(threadp->cond), NULL)) {
! threadp->status = _THREAD_FAILED;
! continue;
! }
! threadp->req = NULL;
! threadp->donereq = NULL;
! if (pthread_create(&(threadp->thread), &globattr, aio_thread_loop,
threadp)) {
! fprintf(stderr, "Thread creation failed\n");
! threadp->status = _THREAD_FAILED;
! continue;
! }
! threadp->next = wait_threads;
! wait_threads = threadp;
}
+
aio_initialised = 1;
}
! static void *
! aio_thread_loop(void *ptr)
{
! aio_thread_t *threadp = (aio_thread_t *) ptr;
! aio_request_t *request;
! struct timespec abstime;
! int ret;
! sigset_t new;
! /* Make sure to ignore signals which may possibly get sent to the parent
*/
! /* squid thread. Causes havoc with mutex's and condition waits
otherwise*/
!
! sigemptyset(&new);
! sigaddset(&new, SIGPIPE);
! sigaddset(&new, SIGCHLD);
! sigaddset(&new, SIGUSR1);
! sigaddset(&new, SIGUSR2);
! sigaddset(&new, SIGHUP);
! sigaddset(&new, SIGTERM);
! sigaddset(&new, SIGINT);
! sigaddset(&new, SIGALRM);
! pthread_sigmask(SIG_BLOCK, &new, NULL);
!
! while(1) {
! /* BELOW is done because Solaris 2.5.1 doesn't support semaphores!!! */
! /* Use timed wait to avoid race where thread context switches after */
! /* threadp->status gets set but before the condition wait happens. */
! /* In that case, a race occurs when the parent signals the condition */
! /* but this thread will never receive it. Recheck every 2-3 secs. */
! /* Also provides bonus of keeping thread contexts hot in CPU cache */
! /* (ie. faster thread reactions) at slight expense of CPU time. */
! while(threadp->req == NULL) {
! abstime.tv_sec = squid_curtime + 3;
! abstime.tv_nsec = 0;
! threadp->status = _THREAD_WAITING;
! ret = pthread_cond_timedwait(&(threadp->cond),
! &(threadp->mutex),
! &abstime);
! }
! request = threadp->req;
! switch(request->request_type) {
! case _AIO_OP_OPEN:
! aio_thread_open(threadp);
break;
! case _AIO_OP_READ:
! aio_thread_read(threadp);
! break;
! case _AIO_OP_WRITE:
! aio_thread_write(threadp);
! break;
! case _AIO_OP_CLOSE:
! aio_thread_close(threadp);
! break;
! case _AIO_OP_UNLINK:
! aio_thread_unlink(threadp);
! break;
! #if 0
! /* Opendir not implemented yet */
! case _AIO_OP_OPENDIR:
! aio_thread_opendir(threadp);
! break;
! #endif
! case _AIO_OP_STAT:
! aio_thread_stat(threadp);
! break;
! default:
! threadp->donereq->ret = -1;
! threadp->donereq->err = EINVAL;
! break;
! }
! threadp->req = NULL;
! } /* while */
! } /* aio_thread_loop */
! static aio_request_t *
! aio_alloc_request()
! {
! aio_request_t *req;
!
! if((req = free_requests) != NULL) {
! free_requests = req->next;
! num_free_requests--;
! return req;
! }
! return (aio_request_t *) xmalloc(sizeof(aio_request_t));
! } /* aio_alloc_request */
static void
! aio_free_request(aio_request_t *req)
{
! /* Below doesn't have to be NUMTHREADS but it's a kinda cute value since
*/
! /* it reflects the sort of load the squid server will experience. A */
! /* higher load will mean a need for more threads, which will in turn
mean */
! /* a need for a bigger free request pool. */
! if(num_free_requests >= NUMTHREADS) {
! xfree(req);
return;
! }
! req->next = free_requests;
! free_requests = req;
! num_free_requests++;
! } /* aio_free_request */
!
!
! static void
! aio_do_request(aio_request_t *requestp)
! {
! aio_thread_t *threadp;
!
! if(wait_threads == NULL && busy_threads_head == NULL) {
! fprintf(stderr, "PANIC: No threads to service requests with!\n");
! exit(-1);
! }
!
! aio_queue_request(requestp);
! aio_process_request_queue();
! } /* aio_do_request */
!
!
! static void
! aio_queue_request(aio_request_t *requestp)
! {
! aio_request_t *rp;
! static int last_warn = 0;
! int i;
!
! if(request_queue_head == NULL) {
! request_queue_head = requestp;
! request_queue_tail = requestp;
} else {
! request_queue_tail->next = requestp;
! request_queue_tail = requestp;
! }
! requestp->next = NULL;
! if(++request_queue_len > NUMTHREADS) {
! if(squid_curtime > (last_warn + 15)) {
! debug(43, 1) ("aio_queue_request: WARNING - Async request queue
growing: Length = %d\n", request_queue_len);
! debug(43, 1) ("aio_queue_request: Perhaps you should increase
NUMTHREADS in aiops.c\n");
! debug(43, 1) ("aio_queue_request: First %d items on request queue\n",
NUMTHREADS);
! rp = request_queue_head;
! for(i = 1; i <= NUMTHREADS; i++) {
! switch(rp->request_type) {
! case _AIO_OP_OPEN:
! debug(43, 1) ("aio_queue_request: %d : open -> %s\n", i, rp->path);
! break;
! case _AIO_OP_READ:
! debug(43, 1) ("aio_queue_request: %d : read -> FD = %d\n", i, rp->fd);
! break;
! case _AIO_OP_WRITE:
! debug(43, 1) ("aio_queue_request: %d : write -> FD = %d\n", i, rp->fd);
! break;
! case _AIO_OP_CLOSE:
! debug(43, 1) ("aio_queue_request: %d : close -> FD = %d\n", i, rp->fd);
! break;
! case _AIO_OP_UNLINK:
! debug(43, 1) ("aio_queue_request: %d : unlink -> %s\n", i, rp->path);
! break;
! case _AIO_OP_STAT:
! debug(43, 1) ("aio_queue_request: %d : stat -> %s\n", i, rp->path);
! break;
! default:
! debug(43, 1) ("aio_queue_request: %d : Unimplemented request type:
%d\n", i, rp->request_type);
! break;
! }
! if((rp = rp->next) == NULL)
! break;
! }
! last_warn = squid_curtime;
! }
! }
! if(request_queue_len > RIDICULOUS_LENGTH) {
! debug(43, 0) ("aio_queue_request: Async request queue growing
uncontrollably!\n");
! debug(43, 0) ("aio_queue_request: Possible infinite loop somewhere in
squid. Restarting...\n");
! abort();
! }
! } /* aio_queue_request */
!
!
! static void
! aio_process_request_queue()
! {
! aio_thread_t *threadp;
! aio_request_t *requestp;
!
! for(;;) {
! if(wait_threads == NULL || request_queue_head == NULL)
return;
!
! requestp = request_queue_head;
! if((request_queue_head = request_queue_head->next) == NULL)
! request_queue_tail = NULL;
! request_queue_len--;
!
! if(requestp->cancelled) {
! aio_cleanup_request(requestp);
! continue;
! }
!
! threadp = wait_threads;
! wait_threads = wait_threads->next;
!
! threadp->req = requestp;
! threadp->donereq = requestp;
! if(busy_threads_head != NULL)
! busy_threads_tail->next = threadp;
! else
! busy_threads_head = threadp;
! busy_threads_tail = threadp;
! threadp->next = NULL;
!
! threadp->status = _THREAD_BUSY;
! pthread_cond_signal(&(threadp->cond));
}
! } /* aio_process_request_queue */
static void
! aio_cleanup_request(aio_request_t *requestp)
{
! aio_result_t *resultp = requestp->resultp;
! int cancelled = requestp->cancelled;
! /* Free allocated structures and copy data back to user space if the */
! /* request hasn't been cancelled */
! switch (requestp->request_type) {
! case _AIO_OP_STAT:
! if(!cancelled && requestp->ret == 0)
! xmemcpy(requestp->statp, requestp->tmpstatp, sizeof(struct stat));
! xfree(requestp->tmpstatp);
case _AIO_OP_OPEN:
case _AIO_OP_UNLINK:
case _AIO_OP_OPENDIR:
! xfree(requestp->path);
! break;
! case _AIO_OP_READ:
! if(!cancelled && requestp->ret > 0)
! xmemcpy(requestp->bufferp, requestp->tmpbufp, requestp->ret);
! case _AIO_OP_WRITE:
! xfree(requestp->tmpbufp);
break;
default:
break;
}
! if(!cancelled) {
! resultp->aio_return = requestp->ret;
! resultp->aio_errno = requestp->err;
! }
! aio_free_request(requestp);
! } /* aio_cleanup_request */
int
aio_cancel(aio_result_t * resultp)
{
aio_thread_t *threadp;
+ aio_request_t *requestp;
int ret;
! for (threadp = busy_threads_head; threadp != NULL; threadp = threadp->
next)
! if(threadp->donereq->resultp == resultp)
! break;
! if (threadp != NULL) {
! threadp->donereq->cancelled = 1;
! return 0;
}
! for(requestp=request_queue_head; requestp!=NULL; requestp = requestp->
next)
! if(requestp->resultp == resultp)
! break;
! if(requestp != NULL) {
! requestp->cancelled = 1;
! return 0;
! }
! errno = ENOENT;
! return -1;
! } /* aio_cancel */
int
aio_open(const char *path, int oflag, mode_t mode, aio_result_t * resultp)
{
! aio_request_t *requestp;
int len;
if (!aio_initialised)
aio_init();
! if ((requestp = aio_alloc_request()) == NULL) {
errno = ENOMEM;
return -1;
}
len = strlen(path) + 1;
! if ((requestp->path = (char *) xmalloc(len)) == NULL) {
! aio_free_request(requestp);
errno = ENOMEM;
return -1;
}
! strncpy(requestp->path, path, len);
! requestp->oflag = oflag;
! requestp->mode = mode;
! requestp->resultp = resultp;
! requestp->request_type = _AIO_OP_OPEN;
! requestp->cancelled = 0;
!
! aio_do_request(requestp);
return 0;
}
! static void
! aio_thread_open(aio_thread_t *threadp)
{
! aio_request_t *requestp = threadp->req;
! requestp->ret = open(requestp->path, requestp->oflag, requestp->mode);
! requestp->err = errno;
}
int
aio_read(int fd, char *bufp, int bufs, off_t offset, int whence,
aio_result_t * resultp)
{
! aio_request_t *requestp;
if (!aio_initialised)
aio_init();
! if ((requestp = aio_alloc_request()) == NULL) {
errno = ENOMEM;
return -1;
}
! requestp->fd = fd;
! requestp->bufferp = bufp;
! if ((requestp->tmpbufp = (char *) xmalloc(bufs)) == NULL) {
! aio_free_request(requestp);
! errno = ENOMEM;
return -1;
}
+ requestp->buflen = bufs;
+ requestp->offset = offset;
+ requestp->whence = whence;
+ requestp->resultp = resultp;
+ requestp->request_type = _AIO_OP_READ;
+ requestp->cancelled = 0;
+
+ aio_do_request(requestp);
return 0;
}
! static void
! aio_thread_read(aio_thread_t *threadp)
{
! aio_request_t *requestp = threadp->req;
! lseek(requestp->fd, requestp->offset, requestp->whence);
! requestp->ret = read(requestp->fd, requestp->tmpbufp, requestp->buflen);
! requestp->err = errno;
}
int
aio_write(int fd, char *bufp, int bufs, off_t offset, int whence,
aio_result_t * resultp)
{
! aio_request_t *requestp;
if (!aio_initialised)
aio_init();
! if ((requestp = aio_alloc_request()) == NULL) {
errno = ENOMEM;
return -1;
}
! requestp->fd = fd;
! if ((requestp->tmpbufp = (char *) xmalloc(bufs)) == NULL) {
! aio_free_request(requestp);
! errno = ENOMEM;
return -1;
}
+ xmemcpy(requestp->tmpbufp, bufp, bufs);
+ requestp->buflen = bufs;
+ requestp->offset = offset;
+ requestp->whence = whence;
+ requestp->resultp = resultp;
+ requestp->request_type = _AIO_OP_WRITE;
+ requestp->cancelled = 0;
+
+ aio_do_request(requestp);
return 0;
}
! static void
! aio_thread_write(aio_thread_t *threadp)
{
! aio_request_t *requestp = threadp->req;
! requestp->ret = write(requestp->fd, requestp->tmpbufp, requestp->buflen);
! requestp->err = errno;
}
int
aio_close(int fd, aio_result_t * resultp)
{
! aio_request_t *requestp;
if (!aio_initialised)
aio_init();
! if ((requestp = aio_alloc_request()) == NULL) {
errno = ENOMEM;
return -1;
}
! requestp->fd = fd;
! requestp->resultp = resultp;
! requestp->request_type = _AIO_OP_CLOSE;
! requestp->cancelled = 0;
!
! aio_do_request(requestp);
return 0;
}
! static void
! aio_thread_close(aio_thread_t *threadp)
{
! aio_request_t *requestp = threadp->req;
! requestp->ret = close(requestp->fd);
! requestp->err = errno;
}
int
aio_stat(const char *path, struct stat *sb, aio_result_t * resultp)
{
! aio_request_t *requestp;
int len;
if (!aio_initialised)
aio_init();
! if ((requestp = aio_alloc_request()) == NULL) {
errno = ENOMEM;
return -1;
}
len = strlen(path) + 1;
! if ((requestp->path = (char *) xmalloc(len)) == NULL) {
! aio_free_request(requestp);
errno = ENOMEM;
return -1;
}
! strncpy(requestp->path, path, len);
! requestp->statp = sb;
! if ((requestp->tmpstatp = (struct stat *) xmalloc(sizeof(struct stat)))
== NULL) {
! xfree(requestp->path);
! aio_free_request(requestp);
! errno = ENOMEM;
return -1;
}
+ requestp->resultp = resultp;
+ requestp->request_type = _AIO_OP_STAT;
+ requestp->cancelled = 0;
+
+ aio_do_request(requestp);
return 0;
}
! static void
! aio_thread_stat(aio_thread_t *threadp)
{
! aio_request_t *requestp = threadp->req;
! requestp->ret = stat(requestp->path, requestp->tmpstatp);
! requestp->err = errno;
}
int
aio_unlink(const char *path, aio_result_t * resultp)
{
! aio_request_t *requestp;
int len;
if (!aio_initialised)
aio_init();
! if ((requestp = aio_alloc_request()) == NULL) {
errno = ENOMEM;
return -1;
}
len = strlen(path) + 1;
! if ((requestp->path = (char *) xmalloc(len)) == NULL) {
! aio_free_request(requestp);
errno = ENOMEM;
return -1;
}
! strncpy(requestp->path, path, len);
! requestp->resultp = resultp;
! requestp->request_type = _AIO_OP_UNLINK;
! requestp->cancelled = 0;
!
! aio_do_request(requestp);
return 0;
}
! static void
! aio_thread_unlink(aio_thread_t *threadp)
{
! aio_request_t *requestp = threadp->req;
! requestp->ret = unlink(requestp->path);
! requestp->err = errno;
}
***************
*** 557,569 ****
int
aio_opendir(const char *path, aio_result_t * resultp)
{
return -1;
}
static void *
! aio_thread_opendir(void *ptr)
{
! aio_thread_t *threadp = (aio_thread_t *) ptr;
return threadp;
}
#endif
--- 699,722 ----
int
aio_opendir(const char *path, aio_result_t * resultp)
{
+ aio_request_t *requestp;
+ int len;
+
+ if (!aio_initialised)
+ aio_init();
+ if ((requestp = aio_alloc_request()) == NULL) {
+ errno = ENOMEM;
+ return -1;
+ }
return -1;
}
static void *
! aio_thread_opendir(aio_thread_t *threadp)
{
! aio_request_t *requestp = threadp->req;
! aio_result_t *resultp = requestp->resultp;
!
return threadp;
}
#endif
***************
*** 572,629 ****
aio_result_t *
aio_poll_done()
{
aio_thread_t *threadp;
aio_result_t *resultp;
! for (threadp = used_threads; threadp != NULL; threadp = threadp->next) {
debug(43, 3) ("%d: %d -> %d\n",
threadp->thread,
! threadp->operation,
threadp->status);
! if (threadp->status == _THREAD_DONE)
break;
}
if (threadp == NULL)
return NULL;
! pthread_join(threadp->thread, NULL);
! resultp = threadp->resultp;
! aio_debug(threadp);
! debug(43, 3) ("DONE: %d -> %d\n",
! resultp->aio_return,
! resultp->aio_errno);
! aio_cleanup_and_free(threadp);
return resultp;
! }
static void
! aio_debug(aio_thread_t * threadp)
{
! aio_open_d *od;
! aio_read_d *rd;
! aio_write_d *wd;
! aio_close_d *cd;
! aio_unlink_d *ud;
!
! switch (threadp->operation) {
case _AIO_OP_OPEN:
! od = (aio_open_d *) threadp->aiodp;
! debug(43, 3) ("OPEN of %s to FD %d\n", od->path, threadp->resultp->
aio_return);
break;
case _AIO_OP_READ:
! rd = (aio_read_d *) threadp->aiodp;
! debug(43, 3) ("READ on fd: %d\n", rd->fd);
break;
case _AIO_OP_WRITE:
! wd = (aio_write_d *) threadp->aiodp;
! debug(43, 3) ("WRITE on fd: %d\n", wd->fd);
break;
case _AIO_OP_CLOSE:
! cd = (aio_close_d *) threadp->aiodp;
! debug(43, 3) ("CLOSE of fd: %d\n", cd->fd);
break;
case _AIO_OP_UNLINK:
! ud = (aio_unlink_d *) threadp->aiodp;
! debug(43, 3) ("UNLINK of %s\n", ud->path);
break;
default:
break;
--- 725,794 ----
aio_result_t *
aio_poll_done()
{
+ aio_thread_t *prev;
aio_thread_t *threadp;
+ aio_request_t *requestp;
aio_result_t *resultp;
+ int cancelled;
! AIO_REPOLL:
! prev = NULL;
! threadp = busy_threads_head;
! while(threadp) {
debug(43, 3) ("%d: %d -> %d\n",
threadp->thread,
! threadp->donereq->request_type,
threadp->status);
! if (threadp->status == _THREAD_WAITING)
break;
+ prev = threadp;
+ threadp = threadp->next;
}
if (threadp == NULL)
return NULL;
!
! if(prev == NULL)
! busy_threads_head = busy_threads_head->next;
! else
! prev->next = threadp->next;
!
! if(busy_threads_tail == threadp)
! busy_threads_tail = prev;
!
! requestp = threadp->donereq;
! threadp->donereq = NULL;
! resultp = requestp->resultp;
! aio_debug(requestp);
! debug(43, 3) ("DONE: %d -> %d\n", requestp->ret, requestp->err);
! threadp->next = wait_threads;
! wait_threads = threadp;
! cancelled = requestp->cancelled;
! aio_cleanup_request(requestp);
! aio_process_request_queue();
! if(cancelled)
! goto AIO_REPOLL;
return resultp;
! } /* aio_poll_done */
!
static void
! aio_debug(aio_request_t * requestp)
{
! switch (requestp->request_type) {
case _AIO_OP_OPEN:
! debug(43, 3) ("OPEN of %s to FD %d\n",requestp->path,requestp->ret);
break;
case _AIO_OP_READ:
! debug(43, 3) ("READ on fd: %d\n", requestp->fd);
break;
case _AIO_OP_WRITE:
! debug(43, 3) ("WRITE on fd: %d\n", requestp->fd);
break;
case _AIO_OP_CLOSE:
! debug(43, 3) ("CLOSE of fd: %d\n", requestp->fd);
break;
case _AIO_OP_UNLINK:
! debug(43, 3) ("UNLINK of %s\n", requestp->path);
break;
default:
break;
Only in squid-1.2.slf/src: aiops.c.old
diff -cr squid-1.2.beta11/src/asn.c squid-1.2.slf/src/asn.c
*** squid-1.2.beta11/src/asn.c Sat Jan 3 09:04:36 1998
--- squid-1.2.slf/src/asn.c Thu Jan 29 12:48:40 1998
***************
*** 1,5 ****
/*
! * $Id: asn.c,v 1.11 1998/01/02 22:04:36 wessels Exp $
*
* DEBUG: section 53 AS Number handling
* AUTHOR: Duane Wessels, Kostas Anagnostakis
--- 1,5 ----
/*
! * $Id: asn.c,v 1.1 1998/01/29 01:48:19 slf Exp slf $
*
* DEBUG: section 53 AS Number handling
* AUTHOR: Duane Wessels, Kostas Anagnostakis
***************
*** 215,220 ****
--- 215,221 ----
put_free_4k_page(buf);
return;
} else if (size < 0) {
+ debug(50, 1) ("asHandleReply: Called with size=%d.\n", size);
put_free_4k_page(buf);
return;
}
diff -cr squid-1.2.beta11/src/async_io.c squid-1.2.slf/src/async_io.c
*** squid-1.2.beta11/src/async_io.c Sat Oct 25 03:50:37 1997
--- squid-1.2.slf/src/async_io.c Thu Jan 22 13:27:05 1998
***************
*** 1,6 ****
/*
! * $Id: async_io.c,v 1.20 1997/10/24 17:50:37 wessels Exp $
*
* DEBUG: section 32 Asynchronous Disk I/O
* AUTHOR: Pete Bentley <pete@demon.net>
--- 1,6 ----
/*
! * $Id: async_io.c,v 1.3 1998/01/13 05:38:32 slf Exp slf $
*
* DEBUG: section 32 Asynchronous Disk I/O
* AUTHOR: Pete Bentley <pete@demon.net>
***************
*** 49,54 ****
--- 49,55 ----
AIOCB *done_handler;
void *done_handler_data;
aio_result_t result;
+ void *tag;
} aio_ctrl_t;
***************
*** 67,85 ****
aioInit()
{
int i;
if (initialised)
return;
for (i = 0; i < SQUID_MAXFD; i++) {
! pool[i].next = free_list;
! free_list = &(pool[i]);
}
initialised = 1;
}
void
! aioOpen(const char *path, int oflag, mode_t mode, AIOCB * callback, void
*callback_data)
{
aio_ctrl_t *ctrlp;
int ret;
--- 68,88 ----
aioInit()
{
int i;
+ aio_ctrl_t *node;
if (initialised)
return;
for (i = 0; i < SQUID_MAXFD; i++) {
! node = pool + i;
! node->next = free_list;
! free_list = node;
}
initialised = 1;
}
void
! aioOpen(const char *path, int oflag, mode_t mode, AIOCB * callback, void
*callback_data, void *tag)
{
aio_ctrl_t *ctrlp;
int ret;
***************
*** 93,99 ****
return;
}
ctrlp = free_list;
! ctrlp->fd = -1;
ctrlp->done_handler = callback;
ctrlp->done_handler_data = callback_data;
ctrlp->operation = _AIO_OPEN;
--- 96,103 ----
return;
}
ctrlp = free_list;
! ctrlp->fd = -2;
! ctrlp->tag = tag;
ctrlp->done_handler = callback;
ctrlp->done_handler_data = callback_data;
ctrlp->operation = _AIO_OPEN;
***************
*** 117,129 ****
if (!initialised)
aioInit();
! aioCancel(fd);
if (free_list == NULL) {
close(fd);
return;
}
ctrlp = free_list;
ctrlp->fd = fd;
ctrlp->done_handler = NULL;
ctrlp->done_handler_data = NULL;
ctrlp->operation = _AIO_CLOSE;
--- 121,134 ----
if (!initialised)
aioInit();
! aioCancel(fd, NULL);
if (free_list == NULL) {
close(fd);
return;
}
ctrlp = free_list;
ctrlp->fd = fd;
+ ctrlp->tag = NULL;
ctrlp->done_handler = NULL;
ctrlp->done_handler_data = NULL;
ctrlp->operation = _AIO_CLOSE;
***************
*** 139,159 ****
void
! aioCancel(int fd)
{
aio_ctrl_t *curr;
aio_ctrl_t *prev;
if (!initialised)
aioInit();
! for (prev = NULL, curr = used_list; curr != NULL;) {
! if (curr->fd != fd) {
prev = curr;
curr = curr->next;
- continue;
}
aio_cancel(&(curr->result));
if (prev == NULL)
used_list = curr->next;
else
--- 144,177 ----
void
! aioCancel(int fd, void *tag)
{
aio_ctrl_t *curr;
aio_ctrl_t *prev;
if (!initialised)
aioInit();
! prev = NULL;
! curr = used_list;
! for(;;) {
! while(curr != NULL) {
! if(curr->fd == fd)
! break;
! if(tag != NULL && curr->tag == tag)
! break;
prev = curr;
curr = curr->next;
}
+ if(curr == NULL)
+ break;
+
aio_cancel(&(curr->result));
+ if (curr->done_handler)
+ (curr->done_handler) (curr->done_handler_data, -2, -2);
+ if (curr->operation == _AIO_UNLINK)
+ outunlink--;
+
if (prev == NULL)
used_list = curr->next;
else
***************
*** 194,199 ****
--- 212,218 ----
}
ctrlp = free_list;
ctrlp->fd = fd;
+ ctrlp->tag = NULL;
ctrlp->done_handler = callback;
ctrlp->done_handler_data = callback_data;
ctrlp->operation = _AIO_WRITE;
***************
*** 235,240 ****
--- 254,260 ----
}
ctrlp = free_list;
ctrlp->fd = fd;
+ ctrlp->tag = NULL;
ctrlp->done_handler = callback;
ctrlp->done_handler_data = callback_data;
ctrlp->operation = _AIO_READ;
***************
*** 252,258 ****
} /* aioRead */
void
! aioStat(char *path, struct stat *sb, AIOCB * callback, void *callback_data)
{
aio_ctrl_t *ctrlp;
--- 272,278 ----
} /* aioRead */
void
! aioStat(char *path, struct stat *sb, AIOCB * callback, void *callback_data,
void *tag)
{
aio_ctrl_t *ctrlp;
***************
*** 265,274 ****
return;
}
ctrlp = free_list;
! ctrlp->fd = -1;
ctrlp->done_handler = callback;
ctrlp->done_handler_data = callback_data;
! ctrlp->operation = _AIO_UNLINK;
if (aio_stat(path, sb, &(ctrlp->result)) < 0) {
if (errno == ENOMEM || errno == EAGAIN || errno == EINVAL)
errno = EWOULDBLOCK;
--- 285,295 ----
return;
}
ctrlp = free_list;
! ctrlp->fd = -2;
! ctrlp->tag = tag;
ctrlp->done_handler = callback;
ctrlp->done_handler_data = callback_data;
! ctrlp->operation = _AIO_STAT;
if (aio_stat(path, sb, &(ctrlp->result)) < 0) {
if (errno == ENOMEM || errno == EAGAIN || errno == EINVAL)
errno = EWOULDBLOCK;
***************
*** 302,308 ****
if (free_list == NULL || outunlink > 10)
return;
ctrlp = free_list;
! ctrlp->fd = -1;
ctrlp->done_handler = callback;
ctrlp->done_handler_data = callback_data;
ctrlp->operation = _AIO_UNLINK;
--- 323,329 ----
if (free_list == NULL || outunlink > 10)
return;
ctrlp = free_list;
! ctrlp->fd = -2;
ctrlp->done_handler = callback;
ctrlp->done_handler_data = callback_data;
ctrlp->operation = _AIO_UNLINK;
diff -cr squid-1.2.beta11/src/client_side.c squid-1.2.slf/src/client_side.c
*** squid-1.2.beta11/src/client_side.c Wed Jan 7 05:12:22 1998
--- squid-1.2.slf/src/client_side.c Thu Jan 29 12:53:23 1998
***************
*** 1,6 ****
/*
! * $Id: client_side.c,v 1.193 1998/01/06 18:12:22 wessels Exp $
*
* DEBUG: section 33 Client-side Routines
* AUTHOR: Duane Wessels
--- 1,6 ----
/*
! * $Id: client_side.c,v 1.1 1998/01/29 01:48:57 slf Exp slf $
*
* DEBUG: section 33 Client-side Routines
* AUTHOR: Duane Wessels
***************
*** 244,249 ****
--- 244,254 ----
new_request->http_ver = old_request->http_ver;
new_request->headers = xstrdup(old_request->headers);
new_request->headers_sz = old_request->headers_sz;
+ if (old_request->body) {
+ new_request->body = xmalloc(old_request->body_sz);
+ xmemcpy(new_request->body, old_request->body, old_request->body_sz);
+ new_request->body_sz = old_request->body_sz;
+ }
requestUnlink(old_request);
http->request = requestLink(new_request);
urlCanonical(http->request, http->uri);
***************
*** 277,282 ****
--- 282,289 ----
http->out.offset = 0;
protoDispatch(http->conn->fd, http->entry, http->request);
/* Register with storage manager to receive updates when data comes in.
*/
+ if(entry->store_status == STORE_ABORTED)
+ debug(33, 0) ("clientProcessExpired: entry->swap_status ==
STORE_ABORTED\n");
storeClientCopy(entry,
http->out.offset,
http->out.offset,
***************
*** 336,341 ****
--- 343,350 ----
entry->refcount++;
} else if (mem->reply->code == 0) {
debug(33, 3) ("clientHandleIMSReply: Incomplete headers for '%s'\n", url);
+ if(entry->store_status == STORE_ABORTED)
+ debug(33, 0) ("clientHandleIMSReply: entry->swap_status ==
STORE_ABORTED\n");
storeClientCopy(entry,
http->out.offset + size,
http->out.offset,
***************
*** 383,388 ****
--- 392,403 ----
/* use clientCacheHit() here as the callback because we might
* be swapping in from disk, and the file might not really be
* there */
+ if(entry->store_status == STORE_ABORTED) {
+ debug(33, 0) ("clientHandleIMSReply: IMS swapin failed on aborted
object\n");
+ http->log_type = LOG_TCP_SWAPFAIL_MISS;
+ clientProcessMiss(http);
+ return;
+ }
storeClientCopy(entry,
http->out.offset,
http->out.offset,
***************
*** 840,846 ****
--- 855,864 ----
debug(12, 3) ("clientCacheHit: request aborted\n");
} else {
/* swap in failure */
+ debug(12, 1) ("KARMA: clientCacheHit fail\n");
http->log_type = LOG_TCP_SWAPFAIL_MISS;
+ if (http->entry)
+ storeRelease(http->entry);
clientProcessMiss(http);
}
}
***************
*** 911,916 ****
--- 929,938 ----
*/
debug(12, 3) ("clientSendMoreData: Appending %d bytes after headers\n",
(int) (size - hdrlen));
+ if (((size - hdrlen) + l) > 8192) {
+ debug(0,0) ("Size, hdrlen, l %d, %d, %d\n", size, hdrlen, l);
+ return;
+ }
xmemcpy(newbuf + l, buf + hdrlen, size - hdrlen);
/* replace buf with newbuf */
freefunc(buf);
***************
*** 991,997 ****
httpRequestFree(http);
if ((http = conn->chr) != NULL) {
debug(12, 1) ("clientWriteComplete: FD %d Sending next request\n", fd);
! if (!storeClientCopyPending(http->entry, http))
storeClientCopy(http->entry,
http->out.offset,
http->out.offset,
--- 1013,1021 ----
httpRequestFree(http);
if ((http = conn->chr) != NULL) {
debug(12, 1) ("clientWriteComplete: FD %d Sending next request\n", fd);
! if (!storeClientCopyPending(http->entry, http)) {
! if(entry->store_status == STORE_ABORTED)
! debug(33, 0) ("clientWriteComplete: entry->swap_status ==
STORE_ABORTED\n");
storeClientCopy(http->entry,
http->out.offset,
http->out.offset,
***************
*** 999,1004 ****
--- 1023,1029 ----
get_free_4k_page(),
clientSendMoreData,
http);
+ }
} else {
debug(12, 5) ("clientWriteComplete: FD %d reading next request\n", fd);
fd_note(fd, "Reading next request");
***************
*** 1017,1022 ****
--- 1042,1049 ----
} else {
/* More data will be coming from primary server; register with
* storage manager. */
+ if(entry->store_status == STORE_ABORTED)
+ debug(33, 0) ("clientWriteComplete 2: entry->swap_status ==
STORE_ABORTED\n");
storeClientCopy(entry,
http->out.offset,
http->out.offset,
***************
*** 1037,1043 ****
assert(size <= SM_PAGE_SIZE);
put_free_4k_page(buf);
buf = NULL;
! if (size < 0) {
debug(12, 1) ("clientGetHeadersForIMS: storeClientCopy failed for '%s'\n",
storeKeyText(entry->key));
clientProcessMiss(http);
--- 1064,1070 ----
assert(size <= SM_PAGE_SIZE);
put_free_4k_page(buf);
buf = NULL;
! if (size < 0 || entry->store_status == STORE_ABORTED) {
debug(12, 1) ("clientGetHeadersForIMS: storeClientCopy failed for '%s'\n",
storeKeyText(entry->key));
clientProcessMiss(http);
***************
*** 1049,1054 ****
--- 1076,1083 ----
return;
}
/* All headers are not yet available, wait for more data */
+ if(entry->store_status == STORE_ABORTED)
+ debug(33, 0) ("clientGetHeadersForIMS: entry->swap_status ==
STORE_ABORTED\n");
storeClientCopy(entry,
http->out.offset + size,
http->out.offset,
***************
*** 1083,1088 ****
--- 1112,1119 ----
http->log_type = LOG_TCP_IMS_HIT;
entry->refcount++;
if (modifiedSince(entry, http->request)) {
+ if(entry->store_status == STORE_ABORTED)
+ debug(33, 0) ("clientGetHeadersForIMS 2: entry->swap_status ==
STORE_ABORTED\n");
storeClientCopy(entry,
http->out.offset,
http->out.offset,
***************
*** 1236,1241 ****
--- 1267,1274 ----
case LOG_TCP_NEGATIVE_HIT:
case LOG_TCP_MEM_HIT:
entry->refcount++; /* HIT CASE */
+ if(entry->store_status == STORE_ABORTED)
+ debug(33, 0) ("clientProcessRequest: entry->swap_status ==
STORE_ABORTED\n");
storeClientCopy(entry,
http->out.offset,
http->out.offset,
***************
*** 1245,1250 ****
--- 1278,1285 ----
http);
break;
case LOG_TCP_IMS_MISS:
+ if(entry->store_status == STORE_ABORTED)
+ debug(33, 0) ("clientProcessRequest 2: entry->swap_status ==
STORE_ABORTED\n");
storeClientCopy(entry,
http->out.offset,
http->out.offset,
diff -cr squid-1.2.beta11/src/comm.c squid-1.2.slf/src/comm.c
*** squid-1.2.beta11/src/comm.c Tue Jan 6 13:47:14 1998
--- squid-1.2.slf/src/comm.c Thu Jan 29 13:25:15 1998
***************
*** 1,6 ****
/*
! * $Id: comm.c,v 1.221 1998/01/06 02:47:14 wessels Exp $
*
* DEBUG: section 5 Socket Functions
* AUTHOR: Harvest Derived
--- 1,6 ----
/*
! * $Id: comm.c,v 1.3 1998/01/14 23:47:45 slf Exp slf $
*
* DEBUG: section 5 Socket Functions
* AUTHOR: Harvest Derived
***************
*** 328,333 ****
--- 328,337 ----
cs->locks--;
if (ia == NULL) {
debug(5, 3) ("commConnectDnsHandle: Unknown host: %s\n", cs->host);
+ if (!dns_error_message) {
+ dns_error_message = "Unknown DNS error";
+ debug(5,1) ("commConnectDnsHandle: Bad dns_error_message\n");
+ }
assert(dns_error_message != NULL);
commConnectCallback(cs, COMM_ERR_DNS);
return;
***************
*** 621,626 ****
--- 625,632 ----
F = &fd_table[fd];
if (EBIT_TEST(F->flags, FD_CLOSING))
return;
+ if(shutdown_pending && (!F->open || F->type == FD_FILE))
+ return;
assert(F->open);
assert(F->type != FD_FILE);
EBIT_SET(F->flags, FD_CLOSING);
***************
*** 733,746 ****
if (((revents = pfds[i].revents) == 0) || ((fd = pfds[i].fd) == -1))
continue;
if (revents & (POLLRDNORM | POLLIN | POLLHUP | POLLERR)) {
! hdl = fd_table[fd].read_handler;
! fd_table[fd].read_handler = NULL;
! hdl(fd, fd_table[fd].read_data);
}
if (revents & (POLLWRNORM | POLLOUT | POLLHUP | POLLERR)) {
! hdl = fd_table[fd].write_handler;
! fd_table[fd].write_handler = NULL;
! hdl(fd, fd_table[fd].write_data);
}
}
/* TO FIX: repoll ICP connection here */
--- 739,756 ----
if (((revents = pfds[i].revents) == 0) || ((fd = pfds[i].fd) == -1))
continue;
if (revents & (POLLRDNORM | POLLIN | POLLHUP | POLLERR)) {
! if(hdl = fd_table[fd].read_handler) {
! fd_table[fd].read_handler = NULL;
! hdl(fd, fd_table[fd].read_data);
! } else
! debug(5,1) ("comm_poll_incoming: NULL read handler\n");
}
if (revents & (POLLWRNORM | POLLOUT | POLLHUP | POLLERR)) {
! if(hdl = fd_table[fd].write_handler) {
! fd_table[fd].write_handler = NULL;
! hdl(fd, fd_table[fd].write_data);
! } else
! debug(5,1) ("comm_poll_incoming: NULL write handler\n");
}
}
/* TO FIX: repoll ICP connection here */
***************
*** 801,814 ****
for (i = 0; i < N; i++) {
fd = fds[i];
if (FD_ISSET(fd, &read_mask)) {
! hdl = fd_table[fd].read_handler;
! fd_table[fd].read_handler = NULL;
! hdl(fd, fd_table[fd].read_data);
}
if (FD_ISSET(fd, &write_mask)) {
! hdl = fd_table[fd].write_handler;
! fd_table[fd].write_handler = NULL;
! hdl(fd, fd_table[fd].write_data);
}
}
}
--- 811,828 ----
for (i = 0; i < N; i++) {
fd = fds[i];
if (FD_ISSET(fd, &read_mask)) {
! if(hdl = fd_table[fd].read_handler) {
! fd_table[fd].read_handler = NULL;
! hdl(fd, fd_table[fd].read_data);
! } else
! debug(5,1) ("comm_select_incoming: NULL read handler\n");
}
if (FD_ISSET(fd, &write_mask)) {
! if(hdl = fd_table[fd].write_handler) {
! fd_table[fd].write_handler = NULL;
! hdl(fd, fd_table[fd].write_data);
! } else
! debug(5,1) ("comm_select_incoming: NULL write handler\n");
}
}
}
***************
*** 864,869 ****
--- 878,886 ----
else
setSocketShutdownLifetimes(1);
}
+ #if USE_ASYNC_IO
+ aioCheckCallbacks();
+ #endif
nfds = 0;
maxfd = Biggest_FD + 1;
for (i = 0; i < maxfd; i++) {
***************
*** 887,893 ****
return COMM_SHUTDOWN;
#if USE_ASYNC_IO
poll_time = sec > 0 ? 50 : 0;
- aioCheckCallbacks();
#else
poll_time = sec > 0 ? 1000 : 0;
#endif
--- 904,909 ----
***************
*** 961,966 ****
--- 977,984 ----
F->timeout_handler = NULL;
F->read_handler = NULL;
F->write_handler = NULL;
+ if(F->open != 0)
+ fd_close(fd);
}
lastinc = polledinc;
}
***************
*** 999,1004 ****
--- 1017,1027 ----
#if !ALARM_UPDATES_TIME
getCurrentTime();
#endif
+
+ #if USE_ASYNC_IO
+ aioCheckCallbacks();
+ #endif
+
FD_ZERO(&readfds);
FD_ZERO(&writefds);
if (shutdown_pending || reconfigure_pending) {
***************
*** 1031,1039 ****
debug(5, 2) ("comm_select: Still waiting on %d FDs\n", nfds);
if (nfds == 0)
return COMM_SHUTDOWN;
- #if USE_ASYNC_IO
- aioCheckCallbacks();
- #endif
for (;;) {
poll_time.tv_sec = sec > 0 ? 1 : 0;
poll_time.tv_usec = 0;
--- 1054,1059 ----
***************
*** 1394,1399 ****
--- 1414,1427 ----
CommWriteStateData *state = NULL;
debug(5, 5) ("comm_write: FD %d: sz %d: hndl %p: data %p.\n",
fd, size, handler, handler_data);
+ if (fd_table[fd].rwstate) {
+ debug(5,1) ("comm_write: fd_table[%d].rwstate != NULL", fd);
+ state = fd_table[fd].rwstate;
+ debug(5,1) ("comm_write: %d'%s',(%d,%d)'%s'\n", size, buf, state->size,
+ state->offset, state->buf);
+ safe_free(fd_table[fd].rwstate);
+ fd_table[fd].rwstate = NULL;
+ }
assert(fd_table[fd].rwstate == NULL);
state = xcalloc(1, sizeof(CommWriteStateData));
state->buf = buf;
diff -cr squid-1.2.beta11/src/disk.c squid-1.2.slf/src/disk.c
*** squid-1.2.beta11/src/disk.c Sat Nov 15 04:21:17 1997
--- squid-1.2.slf/src/disk.c Fri Jan 30 10:17:37 1998
***************
*** 1,6 ****
/*
! * $Id: disk.c,v 1.95 1997/11/14 17:21:17 wessels Exp $
*
* DEBUG: section 6 Disk I/O Routines
* AUTHOR: Harvest Derived
--- 1,6 ----
/*
! * $Id: disk.c,v 1.4 1998/01/14 23:47:12 slf Exp $
*
* DEBUG: section 6 Disk I/O Routines
* AUTHOR: Harvest Derived
***************
*** 135,141 ****
/* Open a disk file. Return a file descriptor */
int
! file_open(const char *path, int mode, FOCB * callback, void *callback_data)
{
int fd;
open_ctrl_t *ctrlp;
--- 135,141 ----
/* Open a disk file. Return a file descriptor */
int
! file_open(const char *path, int mode, FOCB * callback, void *callback_data,
void *tag)
{
int fd;
open_ctrl_t *ctrlp;
***************
*** 152,158 ****
/* Open file */
#if USE_ASYNC_IO
if (callback != NULL) {
! aioOpen(path, mode, 0644, file_open_complete, ctrlp);
return DISK_OK;
}
#endif
--- 152,158 ----
/* Open file */
#if USE_ASYNC_IO
if (callback != NULL) {
! aioOpen(path, mode, 0644, file_open_complete, ctrlp, tag);
return DISK_OK;
}
#endif
***************
*** 168,179 ****
file_open_complete(void *data, int fd, int errcode)
{
open_ctrl_t *ctrlp = (open_ctrl_t *) data;
if (fd < 0) {
errno = errcode;
debug(50, 0) ("file_open: error opening file %s: %s\n", ctrlp->path,
xstrerror());
if (ctrlp->callback)
! (ctrlp->callback) (ctrlp->callback_data, DISK_ERROR);
xfree(ctrlp->path);
xfree(ctrlp);
return;
--- 168,187 ----
file_open_complete(void *data, int fd, int errcode)
{
open_ctrl_t *ctrlp = (open_ctrl_t *) data;
+
+ if(fd == -2 && errcode == -2) { /* Cancelled - clean up */
+ if (ctrlp->callback)
+ (ctrlp->callback) (ctrlp->callback_data, fd, errcode);
+ xfree(ctrlp->path);
+ xfree(ctrlp);
+ return;
+ }
if (fd < 0) {
errno = errcode;
debug(50, 0) ("file_open: error opening file %s: %s\n", ctrlp->path,
xstrerror());
if (ctrlp->callback)
! (ctrlp->callback) (ctrlp->callback_data, DISK_ERROR, errcode);
xfree(ctrlp->path);
xfree(ctrlp);
return;
***************
*** 182,188 ****
commSetCloseOnExec(fd);
fd_open(fd, FD_FILE, ctrlp->path);
if (ctrlp->callback)
! (ctrlp->callback) (ctrlp->callback_data, fd);
xfree(ctrlp->path);
xfree(ctrlp);
}
--- 190,196 ----
commSetCloseOnExec(fd);
fd_open(fd, FD_FILE, ctrlp->path);
if (ctrlp->callback)
! (ctrlp->callback) (ctrlp->callback_data, fd, errcode);
xfree(ctrlp->path);
xfree(ctrlp);
}
***************
*** 192,198 ****
file_close(int fd)
{
fde *F = &fd_table[fd];
! assert(fd >= 0);
assert(F->open);
if (EBIT_TEST(F->flags, FD_WRITE_DAEMON)) {
EBIT_SET(F->flags, FD_CLOSE_REQUEST);
--- 200,209 ----
file_close(int fd)
{
fde *F = &fd_table[fd];
! if(fd < 0) {
! debug(6, 0) ("file_close: FD less than zero: %d\n", fd);
! return;
! }
assert(F->open);
if (EBIT_TEST(F->flags, FD_WRITE_DAEMON)) {
EBIT_SET(F->flags, FD_CLOSE_REQUEST);
***************
*** 202,214 ****
EBIT_SET(F->flags, FD_CLOSE_REQUEST);
return;
}
- fd_close(fd);
- debug(6, 5) ("file_close: FD %d\n", fd);
#if USE_ASYNC_IO
aioClose(fd);
#else
close(fd);
#endif
}
--- 213,225 ----
EBIT_SET(F->flags, FD_CLOSE_REQUEST);
return;
}
#if USE_ASYNC_IO
aioClose(fd);
#else
close(fd);
#endif
+ debug(6, 5) ("file_close: FD %d\n", fd);
+ fd_close(fd);
}
***************
*** 276,291 ****
dwrite_q *q = fdd->write_q;
int status = DISK_OK;
errno = errcode;
safe_free(data);
- fd_bytes(fd, len, FD_WRITE);
if (q == NULL) /* Someone aborted then write completed */
return;
if (len < 0) {
if (!ignoreErrno(errno)) {
status = errno == ENOSPC ? DISK_NO_SPACE_LEFT : DISK_ERROR;
debug(50, 1) ("diskHandleWrite: FD %d: disk write error: %s\n",
fd, xstrerror());
! if (fdd->wrt_handle == NULL) {
/* FLUSH PENDING BUFFERS */
do {
fdd->write_q = q->next;
--- 287,314 ----
dwrite_q *q = fdd->write_q;
int status = DISK_OK;
errno = errcode;
+
safe_free(data);
if (q == NULL) /* Someone aborted then write completed */
return;
+
+ if(len == -2 && errcode == -2) { /* Write cancelled - cleanup */
+ do {
+ fdd->write_q = q->next;
+ if (q->free)
+ (q->free) (q->buf);
+ safe_free(q);
+ } while ((q = fdd->write_q));
+ return;
+ }
+
+ fd_bytes(fd, len, FD_WRITE);
if (len < 0) {
if (!ignoreErrno(errno)) {
status = errno == ENOSPC ? DISK_NO_SPACE_LEFT : DISK_ERROR;
debug(50, 1) ("diskHandleWrite: FD %d: disk write error: %s\n",
fd, xstrerror());
! if (fdd->wrt_handle == NULL || status != DISK_NO_SPACE_LEFT) {
/* FLUSH PENDING BUFFERS */
do {
fdd->write_q = q->next;
***************
*** 300,305 ****
--- 323,331 ----
if (q != NULL) {
/* q might become NULL from write failure above */
q->cur_offset += len;
+ if (q->cur_offset > q->len)
+ debug(50, 1) ("diskHandleWriteComplete: q->cur_offset > q->len (%p,%d,
%d, %d FD %d)\n",
+ q, q->cur_offset, q->len, len, fd);
assert(q->cur_offset <= q->len);
if (q->cur_offset == q->len) {
/* complete write */
***************
*** 339,344 ****
--- 365,372 ----
{
dwrite_q *wq = NULL;
fde *F = &fd_table[fd];
+ if(shutdown_pending && (fd < 0 || !F->open))
+ return;
assert(fd >= 0);
assert(F->open);
/* if we got here. Caller is eligible to write. */
***************
*** 411,417 ****
--- 439,453 ----
int fd = ctrlp->fd;
int rc = DISK_OK;
errno = errcode;
+
xfree(data);
+
+ if(len == -2 && errcode == -2) { /* Read cancelled - cleanup */
+ cbdataUnlock(ctrl_dat->client_data);
+ safe_free(ctrl_dat);
+ return;
+ }
+
fd_bytes(fd, len, FD_READ);
if (len < 0) {
if (ignoreErrno(errno)) {
diff -cr squid-1.2.beta11/src/errorpage.c squid-1.2.slf/src/errorpage.c
*** squid-1.2.beta11/src/errorpage.c Tue Jan 6 16:12:08 1998
--- squid-1.2.slf/src/errorpage.c Fri Jan 9 14:42:17 1998
***************
*** 1,6 ****
/*
! * $Id: errorpage.c,v 1.114 1998/01/06 05:12:08 wessels Exp $
*
* DEBUG: section 4 Error Generation
* AUTHOR: Duane Wessels
--- 1,6 ----
/*
! * $Id: errorpage.c,v 1.2 1998/01/09 03:42:14 slf Exp slf $
*
* DEBUG: section 4 Error Generation
* AUTHOR: Duane Wessels
***************
*** 64,70 ****
for (i = ERR_NONE + 1; i < ERR_MAX; i++) {
snprintf(path, MAXPATHLEN, "%s/%s",
Config.errorDirectory, err_type_str[i]);
! fd = file_open(path, O_RDONLY, NULL, NULL);
if (fd < 0) {
debug(4, 0) ("errorInitialize: %s: %s\n", path, xstrerror());
fatal("Failed to open error text file");
--- 64,70 ----
for (i = ERR_NONE + 1; i < ERR_MAX; i++) {
snprintf(path, MAXPATHLEN, "%s/%s",
Config.errorDirectory, err_type_str[i]);
! fd = file_open(path, O_RDONLY, NULL, NULL, NULL);
if (fd < 0) {
debug(4, 0) ("errorInitialize: %s: %s\n", path, xstrerror());
fatal("Failed to open error text file");
diff -cr squid-1.2.beta11/src/fd.c squid-1.2.slf/src/fd.c
*** squid-1.2.beta11/src/fd.c Tue Dec 2 11:17:34 1997
--- squid-1.2.slf/src/fd.c Thu Jan 29 13:07:32 1998
***************
*** 1,6 ****
/*
! * $Id: fd.c,v 1.16 1997/12/02 00:17:34 wessels Exp $
*
* DEBUG: section 51 Filedescriptor Functions
* AUTHOR: Duane Wessels
--- 1,6 ----
/*
! * $Id: fd.c,v 1.1 1998/01/14 23:50:01 slf Exp slf $
*
* DEBUG: section 51 Filedescriptor Functions
* AUTHOR: Duane Wessels
***************
*** 40,51 ****
return;
assert(fd < Squid_MaxFD);
if (fd > Biggest_FD) {
! assert(status == FD_OPEN);
Biggest_FD = fd;
return;
}
/* if we are here, then fd == Biggest_FD */
! assert(status == FD_CLOSE);
while (fd_table[Biggest_FD].open != FD_OPEN)
Biggest_FD--;
}
--- 40,53 ----
return;
assert(fd < Squid_MaxFD);
if (fd > Biggest_FD) {
! if (status != FD_OPEN)
! debug(51,1) ("fdUpdateBiggest: status != FD_OPEN\n");
Biggest_FD = fd;
return;
}
/* if we are here, then fd == Biggest_FD */
! if (status != FD_CLOSE)
! debug(51,1) ("fdUpdateBiggest: status != FD_CLOSE\n");
while (fd_table[Biggest_FD].open != FD_OPEN)
Biggest_FD--;
}
***************
*** 69,74 ****
--- 71,80 ----
{
fde *F = &fd_table[fd];
assert(fd >= 0);
+ if(F->open != 0) {
+ debug(51, 1) ("WARNING: Closing open FD %4d\n", fd);
+ fd_close(fd);
+ }
assert(F->open == 0);
F->type = type;
fdUpdateBiggest(fd, F->open = FD_OPEN);
diff -cr squid-1.2.beta11/src/filemap.c squid-1.2.slf/src/filemap.c
*** squid-1.2.beta11/src/filemap.c Thu Nov 13 10:47:38 1997
--- squid-1.2.slf/src/filemap.c Thu Jan 29 14:58:24 1998
***************
*** 1,5 ****
/*
! * $Id: filemap.c,v 1.21 1997/11/12 23:47:38 wessels Exp $
*
* DEBUG: section 8 Swap File Bitmap
* AUTHOR: Harvest Derived
--- 1,5 ----
/*
! * $Id: filemap.c,v 1.1 1998/01/29 03:56:39 slf Exp slf $
*
* DEBUG: section 8 Swap File Bitmap
* AUTHOR: Harvest Derived
***************
*** 174,179 ****
--- 174,181 ----
int word;
int bit;
int count;
+ if (suggestion > fm->max_n_files)
+ suggestion = 0;
if (!file_map_bit_test(fm, suggestion)) {
fm->last_file_number_allocated = suggestion;
return file_map_bit_set(fm, suggestion);
diff -cr squid-1.2.beta11/src/main.c squid-1.2.slf/src/main.c
*** squid-1.2.beta11/src/main.c Wed Jan 7 05:12:23 1998
--- squid-1.2.slf/src/main.c Fri Jan 30 12:10:36 1998
***************
*** 437,443 ****
--- 437,445 ----
#endif
if (!configured_once) {
+ #if !USE_ASYNC_IO
unlinkdInit();
+ #endif
/* module initialization */
urlInitialize();
stat_init(&HTTPCacheInfo, Config.Log.access);
Only in squid-1.2.slf/src: malloc.c
diff -cr squid-1.2.beta11/src/mime.c squid-1.2.slf/src/mime.c
*** squid-1.2.beta11/src/mime.c Fri Nov 14 16:13:29 1997
--- squid-1.2.slf/src/mime.c Fri Jan 9 14:42:01 1998
***************
*** 1,5 ****
/*
! * $Id: mime.c,v 1.44 1997/11/14 05:13:29 wessels Exp $
*
* DEBUG: section 25 MIME Parsing
* AUTHOR: Harvest Derived
--- 1,5 ----
/*
! * $Id: mime.c,v 1.2 1998/01/09 03:41:57 slf Exp slf $
*
* DEBUG: section 25 MIME Parsing
* AUTHOR: Harvest Derived
***************
*** 404,410 ****
if (storeGet(key))
return;
snprintf(path, MAXPATHLEN, "%s/%s", Config.icons.directory, icon);
! fd = file_open(path, O_RDONLY, NULL, NULL);
if (fd < 0) {
debug(25, 0) ("mimeLoadIconFile: %s: %s\n", path, xstrerror());
return;
--- 404,410 ----
if (storeGet(key))
return;
snprintf(path, MAXPATHLEN, "%s/%s", Config.icons.directory, icon);
! fd = file_open(path, O_RDONLY, NULL, NULL, NULL);
if (fd < 0) {
debug(25, 0) ("mimeLoadIconFile: %s: %s\n", path, xstrerror());
return;
Only in squid-1.2.slf/src: options.h
diff -cr squid-1.2.beta11/src/protos.h squid-1.2.slf/src/protos.h
*** squid-1.2.beta11/src/protos.h Wed Jan 7 05:15:29 1998
--- squid-1.2.slf/src/protos.h Fri Jan 9 11:46:18 1998
***************
*** 39,50 ****
extern aio_result_t *aio_poll_done(void);
! extern void aioCancel(int);
! extern void aioOpen(const char *, int, mode_t, AIOCB *, void *);
extern void aioClose(int);
extern void aioWrite(int, char *, int, AIOCB *, void *);
extern void aioRead(int, char *, int, AIOCB *, void *);
! extern void aioStat(char *, struct stat *, AIOCB *, void *);
extern void aioUnlink(const char *, AIOCB *, void *);
extern void aioCheckCallbacks(void);
--- 39,50 ----
extern aio_result_t *aio_poll_done(void);
! extern void aioCancel(int, void *);
! extern void aioOpen(const char *, int, mode_t, AIOCB *, void *, void *);
extern void aioClose(int);
extern void aioWrite(int, char *, int, AIOCB *, void *);
extern void aioRead(int, char *, int, AIOCB *, void *);
! extern void aioStat(char *, struct stat *, AIOCB *, void *, void *);
extern void aioUnlink(const char *, AIOCB *, void *);
extern void aioCheckCallbacks(void);
***************
*** 120,126 ****
#endif
! extern int file_open(const char *path, int mode, FOCB *, void
*callback_data);
extern void file_close(int fd);
extern int file_write(int fd,
char *buf,
--- 120,126 ----
#endif
! extern int file_open(const char *path, int mode, FOCB *, void
*callback_data, void *tag);
extern void file_close(int fd);
extern int file_write(int fd,
char *buf,
diff -cr squid-1.2.beta11/src/send-announce.c squid-1.2.slf/src/send-announce.c
*** squid-1.2.beta11/src/send-announce.c Wed Nov 5 16:29:35 1997
--- squid-1.2.slf/src/send-announce.c Fri Jan 9 14:41:40 1998
***************
*** 1,6 ****
/*
! * $Id: send-announce.c,v 1.45 1997/11/05 05:29:35 wessels Exp $
*
* DEBUG: section 27 Cache Announcer
* AUTHOR: Duane Wessels
--- 1,6 ----
/*
! * $Id: send-announce.c,v 1.2 1998/01/09 03:41:28 slf Exp slf $
*
* DEBUG: section 27 Cache Announcer
* AUTHOR: Duane Wessels
***************
*** 78,84 ****
strcat(sndbuf, tbuf);
l = strlen(sndbuf);
if ((file = Config.Announce.file) != NULL) {
! fd = file_open(file, O_RDONLY, NULL, NULL);
if (fd > -1 && (n = read(fd, sndbuf + l, BUFSIZ - l - 1)) > 0) {
fd_bytes(fd, n, FD_READ);
l += n;
--- 78,84 ----
strcat(sndbuf, tbuf);
l = strlen(sndbuf);
if ((file = Config.Announce.file) != NULL) {
! fd = file_open(file, O_RDONLY, NULL, NULL, NULL);
if (fd > -1 && (n = read(fd, sndbuf + l, BUFSIZ - l - 1)) > 0) {
fd_bytes(fd, n, FD_READ);
l += n;
Only in squid-1.2.slf/src: squid.conf
Only in squid-1.2.slf/src: srcfiles.tar.gz
Only in squid-1.2.slf/src: srcfiles.tar.gz.uu
diff -cr squid-1.2.beta11/src/stmem.c squid-1.2.slf/src/stmem.c
*** squid-1.2.beta11/src/stmem.c Tue Jan 6 08:18:15 1998
--- squid-1.2.slf/src/stmem.c Thu Jan 29 13:10:05 1998
***************
*** 1,6 ****
/*
! * $Id: stmem.c,v 1.54 1998/01/05 21:18:15 wessels Exp $
*
* DEBUG: section 19 Memory Primitives
* AUTHOR: Harvest Derived
--- 1,6 ----
/*
! * $Id: stmem.c,v 1.1 1998/01/29 02:08:18 slf Exp slf $
*
* DEBUG: section 19 Memory Primitives
* AUTHOR: Harvest Derived
***************
*** 229,234 ****
--- 229,238 ----
/* Seek our way into store */
while ((t_off + p->len) < offset) {
t_off += p->len;
+ if(!p->next) {
+ debug(19,1) ("memCopy: p->next == NULL\n");
+ return 0;
+ }
assert(p->next);
p = p->next;
}
diff -cr squid-1.2.beta11/src/store.c squid-1.2.slf/src/store.c
*** squid-1.2.beta11/src/store.c Tue Jan 6 18:11:55 1998
--- squid-1.2.slf/src/store.c Fri Jan 30 13:41:38 1998
***************
*** 1,6 ****
/*
! * $Id: store.c,v 1.364 1998/01/06 07:11:55 wessels Exp $
*
* DEBUG: section 20 Storeage Manager
* AUTHOR: Harvest Derived
--- 1,6 ----
/*
! * $Id: store.c,v 1.7 1998/01/15 05:43:47 slf Exp slf $
*
* DEBUG: section 20 Storeage Manager
* AUTHOR: Harvest Derived
***************
*** 169,174 ****
--- 169,175 ----
int expcount; /* # objects expired */
int linecount; /* # lines parsed from cache logfile */
int clashcount; /* # swapfile clashes avoided */
+ int cancelcount; /* # objects cancelled */
int dupcount; /* # duplicates purged */
int invalid; /* # bad lines */
int badflags; /* # bad e->flags */
***************
*** 184,190 ****
struct storeCleanList *next;
} storeCleanList;
! typedef void (VCB) (void *);
typedef struct valid_ctrl_t {
struct stat *sb;
--- 185,191 ----
struct storeCleanList *next;
} storeCleanList;
! typedef void (VCB) (void *, int, int);
typedef struct valid_ctrl_t {
struct stat *sb;
***************
*** 234,240 ****
u_num32,
u_num32,
int);
! static void destroy_MemObject(MemObject *);
static void destroy_MemObjectData(MemObject *);
static void destroy_StoreEntry(StoreEntry *);
static void storePurgeMem(StoreEntry *);
--- 235,241 ----
u_num32,
u_num32,
int);
! static void destroy_MemObject(StoreEntry *);
static void destroy_MemObjectData(MemObject *);
static void destroy_StoreEntry(StoreEntry *);
static void storePurgeMem(StoreEntry *);
***************
*** 245,258 ****
static EVH storeDoRebuildFromDisk;
static EVH storeCleanup;
static VCB storeCleanupComplete;
! static void storeValidate(StoreEntry *, VCB *, void *);
static AIOCB storeValidateComplete;
static void storeRebuiltFromDisk(struct storeRebuildState *data);
static unsigned int getKeyCounter(void);
! static void storePutUnusedFileno(int fileno);
static int storeGetUnusedFileno(void);
static void storeCheckSwapOut(StoreEntry * e);
! static void storeSwapoutFileOpened(void *data, int fd);
static int storeCheckCachable(StoreEntry * e);
static int storeKeepInMemory(const StoreEntry *);
static SIH storeClientCopyFileOpened;
--- 246,259 ----
static EVH storeDoRebuildFromDisk;
static EVH storeCleanup;
static VCB storeCleanupComplete;
! static void storeValidate(StoreEntry *, VCB *, void *, void *);
static AIOCB storeValidateComplete;
static void storeRebuiltFromDisk(struct storeRebuildState *data);
static unsigned int getKeyCounter(void);
! static void storePutUnusedFileno(StoreEntry *e);
static int storeGetUnusedFileno(void);
static void storeCheckSwapOut(StoreEntry * e);
! static void storeSwapoutFileOpened(void *data, int fd, int errcode);
static int storeCheckCachable(StoreEntry * e);
static int storeKeepInMemory(const StoreEntry *);
static SIH storeClientCopyFileOpened;
***************
*** 283,288 ****
--- 284,293 ----
static int store_maintain_rate;
static int store_maintain_buckets;
+ /* outstanding cleanup validations */
+
+ static int outvalid = 0;
+
static MemObject *
new_MemObject(const char *url, const char *log_url)
{
***************
*** 315,326 ****
}
static void
! destroy_MemObject(MemObject * mem)
{
debug(20, 3) ("destroy_MemObject: destroying %p\n", mem);
assert(mem->swapout.fd == -1);
destroy_MemObjectData(mem);
meta_data.misc -= strlen(mem->log_url);
assert(mem->clients == NULL);
safe_free(mem->reply);
safe_free(mem->url);
--- 320,336 ----
}
static void
! destroy_MemObject(StoreEntry *e)
{
+ MemObject *mem = e->mem_obj;
debug(20, 3) ("destroy_MemObject: destroying %p\n", mem);
assert(mem->swapout.fd == -1);
destroy_MemObjectData(mem);
meta_data.misc -= strlen(mem->log_url);
+ #if USE_ASYNC_IO
+ while(mem->clients != NULL)
+ storeUnregister(e, mem->clients->callback_data);
+ #endif
assert(mem->clients == NULL);
safe_free(mem->reply);
safe_free(mem->url);
***************
*** 338,344 ****
debug(20, 3) ("destroy_StoreEntry: destroying %p\n", e);
assert(e != NULL);
if (e->mem_obj)
! destroy_MemObject(e->mem_obj);
storeHashDelete(e);
assert(e->key == NULL);
xfree(e);
--- 348,354 ----
debug(20, 3) ("destroy_StoreEntry: destroying %p\n", e);
assert(e != NULL);
if (e->mem_obj)
! destroy_MemObject(e);
storeHashDelete(e);
assert(e->key == NULL);
xfree(e);
***************
*** 438,444 ****
debug(20, 3) ("storePurgeMem: Freeing memory-copy of %s\n",
storeKeyText(e->key));
storeSetMemStatus(e, NOT_IN_MEMORY);
! destroy_MemObject(e->mem_obj);
e->mem_obj = NULL;
if (e->swap_status != SWAPOUT_DONE)
storeRelease(e);
--- 448,454 ----
debug(20, 3) ("storePurgeMem: Freeing memory-copy of %s\n",
storeKeyText(e->key));
storeSetMemStatus(e, NOT_IN_MEMORY);
! destroy_MemObject(e);
e->mem_obj = NULL;
if (e->swap_status != SWAPOUT_DONE)
storeRelease(e);
***************
*** 475,480 ****
--- 485,492 ----
e->lock_count--;
debug(20, 3) ("storeUnlockObject: key '%s' count=%d\n",
storeKeyText(e->key), e->lock_count);
+ if(e->lock_count < 0)
+ e->lock_count = 0;
if (e->lock_count)
return (int) e->lock_count;
if (e->store_status == STORE_PENDING) {
***************
*** 629,643 ****
EBIT_CLR(e->flag, RELEASE_REQUEST);
EBIT_CLR(e->flag, KEY_PRIVATE);
e->ping_status = PING_NONE;
! if (clean) {
! EBIT_SET(e->flag, ENTRY_VALIDATED);
! /* Only set the file bit if we know its a valid entry */
! /* otherwise, set it in the validation procedure */
! storeDirMapBitSet(file_number);
! storeDirUpdateSwapSize(e->swap_file_number, e->object_len, 1);
! } else {
! EBIT_CLR(e->flag, ENTRY_VALIDATED);
! }
return e;
}
--- 641,648 ----
EBIT_CLR(e->flag, RELEASE_REQUEST);
EBIT_CLR(e->flag, KEY_PRIVATE);
e->ping_status = PING_NONE;
! EBIT_CLR(e->flag, ENTRY_VALIDATED);
! storeDirMapBitSet(e->swap_file_number);
return e;
}
***************
*** 659,670 ****
--- 664,680 ----
return 0;
*S = sc->next;
mem->nclients--;
+ sc->disk_op_in_progress = 0;
if (e->store_status == STORE_OK && e->swap_status != SWAPOUT_DONE)
storeCheckSwapOut(e);
if (sc->swapin_fd > -1) {
commSetSelect(sc->swapin_fd, COMM_SELECT_READ, NULL, NULL, 0);
file_close(sc->swapin_fd);
}
+ #if USE_ASYNC_IO
+ else
+ aioCancel(-1, sc);
+ #endif
if ((callback = sc->callback) != NULL) {
/* callback with ssize = -1 to indicate unexpected termination */
debug(20, 3) ("storeUnregister: store_client for %s has a callback\n",
***************
*** 726,739 ****
}
static void
! storeSwapoutFileOpened(void *data, int fd)
{
swapout_ctrl_t *ctrlp = data;
int oldswapstatus = ctrlp->oldswapstatus;
char *swapfilename = ctrlp->swapfilename;
StoreEntry *e = ctrlp->e;
MemObject *mem;
xfree(ctrlp);
assert(e->swap_status == SWAPOUT_OPENING);
if (fd < 0) {
debug(20, 0) ("storeSwapoutFileOpened: Unable to open swapfile: %s\n",
--- 736,754 ----
}
static void
! storeSwapoutFileOpened(void *data, int fd, int errcode)
{
swapout_ctrl_t *ctrlp = data;
int oldswapstatus = ctrlp->oldswapstatus;
char *swapfilename = ctrlp->swapfilename;
StoreEntry *e = ctrlp->e;
MemObject *mem;
+
xfree(ctrlp);
+ if(fd == -2 && errcode == -2) { /* Cancelled - Clean up */
+ xfree(swapfilename);
+ return;
+ }
assert(e->swap_status == SWAPOUT_OPENING);
if (fd < 0) {
debug(20, 0) ("storeSwapoutFileOpened: Unable to open swapfile: %s\n",
***************
*** 761,767 ****
--- 776,784 ----
swapout_ctrl_t *ctrlp;
LOCAL_ARRAY(char, swapfilename, SQUID_MAXPATHLEN);
storeLockObject(e);
+ #if !MONOTONIC_STORE
if ((e->swap_file_number = storeGetUnusedFileno()) < 0)
+ #endif
e->swap_file_number = storeDirMapAllocate();
storeSwapFullPath(e->swap_file_number, swapfilename);
ctrlp = xmalloc(sizeof(swapout_ctrl_t));
***************
*** 772,778 ****
file_open(swapfilename,
O_WRONLY | O_CREAT | O_TRUNC,
storeSwapoutFileOpened,
! ctrlp);
}
static void
--- 789,795 ----
file_open(swapfilename,
O_WRONLY | O_CREAT | O_TRUNC,
storeSwapoutFileOpened,
! ctrlp, e);
}
static void
***************
*** 781,793 ****
StoreEntry *e = data;
MemObject *mem = e->mem_obj;
debug(20, 3) ("storeSwapOutHandle: '%s', len=%d\n", storeKeyText(e->
key), (int) len);
- assert(mem != NULL);
if (flag < 0) {
debug(20, 1) ("storeSwapOutHandle: SwapOut failure (err code = %d).\n",
flag);
e->swap_status = SWAPOUT_NONE;
if (e->swap_file_number > -1) {
! storePutUnusedFileno(e->swap_file_number);
e->swap_file_number = -1;
}
if (flag == DISK_NO_SPACE_LEFT) {
--- 798,817 ----
StoreEntry *e = data;
MemObject *mem = e->mem_obj;
debug(20, 3) ("storeSwapOutHandle: '%s', len=%d\n", storeKeyText(e->
key), (int) len);
if (flag < 0) {
debug(20, 1) ("storeSwapOutHandle: SwapOut failure (err code = %d).\n",
flag);
e->swap_status = SWAPOUT_NONE;
if (e->swap_file_number > -1) {
! #if MONOTONIC_STORE
! #if USE_ASYNC_IO
! safeunlink(storeSwapFullPath(e->swap_file_number, NULL), 1);
! #else
! unlinkdUnlink(storeSwapFullPath(e->swap_file_number, NULL));
! #endif
! #else
! storePutUnusedFileno(e);
! #endif
e->swap_file_number = -1;
}
if (flag == DISK_NO_SPACE_LEFT) {
***************
*** 799,804 ****
--- 823,836 ----
storeSwapOutFileClose(e);
return;
}
+ #if USE_ASYNC_IO
+ if(mem == NULL) {
+ debug(20, 1) ("storeSwapOutHandle: mem == NULL : Cancelling swapout\n");
+ return;
+ }
+ #else
+ assert(mem != NULL);
+ #endif
mem->swapout.done_offset += len;
if (e->store_status == STORE_PENDING || mem->swapout.done_offset < e->
object_len) {
storeCheckSwapOut(e);
***************
*** 850,856 ****
--- 882,897 ----
(int) mem->swapout.queue_offset);
debug(20, 3) ("storeCheckSwapOut: swapout.done_offset = %d\n",
(int) mem->swapout.done_offset);
+ #if USE_ASYNC_IO
+ if(mem->inmem_hi < mem->swapout.queue_offset) {
+ storeAbort(e, 0);
+ assert(EBIT_TEST(e->flag, RELEASE_REQUEST));
+ storeSwapOutFileClose(e);
+ return;
+ }
+ #else
assert(mem->inmem_hi >= mem->swapout.queue_offset);
+ #endif
swapout_size = (size_t) (mem->inmem_hi - mem->swapout.queue_offset);
lowest_offset = storeLowestMemReaderOffset(e);
debug(20, 3) ("storeCheckSwapOut: lowest_offset = %d\n",
***************
*** 859,865 ****
new_mem_lo = lowest_offset;
if (!EBIT_TEST(e->flag, ENTRY_CACHABLE)) {
! assert(EBIT_TEST(e->flag, KEY_PRIVATE));
memFreeDataUpto(mem->data, new_mem_lo);
mem->inmem_lo = new_mem_lo;
return;
--- 900,907 ----
new_mem_lo = lowest_offset;
if (!EBIT_TEST(e->flag, ENTRY_CACHABLE)) {
! if(!EBIT_TEST(e->flag, KEY_PRIVATE))
! debug(20, 0) ("storeCheckSwapOut: Attempt to swap out a non-cacheable
non-private object!\n");
memFreeDataUpto(mem->data, new_mem_lo);
mem->inmem_lo = new_mem_lo;
return;
***************
*** 968,985 ****
}
/* start swapping in */
void
storeSwapInStart(StoreEntry * e, SIH * callback, void *callback_data)
{
swapin_ctrl_t *ctrlp;
assert(e->mem_status == NOT_IN_MEMORY);
- if (!EBIT_TEST(e->flag, ENTRY_VALIDATED)) {
- if (storeDirMapBitTest(e->swap_file_number)) {
- /* someone took our file while we weren't looking */
- callback(-1, callback_data);
- return;
- }
- }
assert(e->swap_status == SWAPOUT_WRITING || e->swap_status ==
SWAPOUT_DONE);
assert(e->swap_file_number >= 0);
assert(e->mem_obj != NULL);
--- 1010,1021 ----
}
/* start swapping in */
+ /* callback_data will become the tag on which the stat/open can be aborted */
void
storeSwapInStart(StoreEntry * e, SIH * callback, void *callback_data)
{
swapin_ctrl_t *ctrlp;
assert(e->mem_status == NOT_IN_MEMORY);
assert(e->swap_status == SWAPOUT_WRITING || e->swap_status ==
SWAPOUT_DONE);
assert(e->swap_file_number >= 0);
assert(e->mem_obj != NULL);
***************
*** 988,1004 ****
ctrlp->callback = callback;
ctrlp->callback_data = callback_data;
if (EBIT_TEST(e->flag, ENTRY_VALIDATED))
! storeSwapInValidateComplete(ctrlp);
else
! storeValidate(e, storeSwapInValidateComplete, ctrlp);
}
static void
! storeSwapInValidateComplete(void *data)
{
swapin_ctrl_t *ctrlp = (swapin_ctrl_t *) data;
StoreEntry *e;
e = ctrlp->e;
assert(e->mem_status == NOT_IN_MEMORY);
if (!EBIT_TEST(e->flag, ENTRY_VALIDATED)) {
--- 1024,1045 ----
ctrlp->callback = callback;
ctrlp->callback_data = callback_data;
if (EBIT_TEST(e->flag, ENTRY_VALIDATED))
! storeSwapInValidateComplete(ctrlp, 0, 0);
else
! storeValidate(e, storeSwapInValidateComplete, ctrlp, callback_data);
}
static void
! storeSwapInValidateComplete(void *data, int retcode, int errcode)
{
swapin_ctrl_t *ctrlp = (swapin_ctrl_t *) data;
StoreEntry *e;
+
+ if(retcode == -2 && errcode == -2) {
+ xfree(ctrlp);
+ return;
+ }
e = ctrlp->e;
assert(e->mem_status == NOT_IN_MEMORY);
if (!EBIT_TEST(e->flag, ENTRY_VALIDATED)) {
***************
*** 1008,1033 ****
return;
}
ctrlp->path = xstrdup(storeSwapFullPath(e->swap_file_number, NULL));
! file_open(ctrlp->path, O_RDONLY, storeSwapInFileOpened, ctrlp);
}
static void
! storeSwapInFileOpened(void *data, int fd)
{
swapin_ctrl_t *ctrlp = (swapin_ctrl_t *) data;
StoreEntry *e = ctrlp->e;
MemObject *mem = e->mem_obj;
assert(mem != NULL);
assert(e->mem_status == NOT_IN_MEMORY);
assert(e->swap_status == SWAPOUT_WRITING || e->swap_status ==
SWAPOUT_DONE);
if (fd < 0) {
! debug(20, 0) ("storeSwapInStartComplete: Failed for '%s'\n", mem->url);
/* Invoke a store abort that should free the memory object */
(ctrlp->callback) (-1, ctrlp->callback_data);
xfree(ctrlp->path);
xfree(ctrlp);
return;
}
debug(20, 5) ("storeSwapInStart: initialized swap file '%s' for '%s'\n",
ctrlp->path, mem->url);
(ctrlp->callback) (fd, ctrlp->callback_data);
--- 1049,1089 ----
return;
}
ctrlp->path = xstrdup(storeSwapFullPath(e->swap_file_number, NULL));
! file_open(ctrlp->path, O_RDONLY, storeSwapInFileOpened, ctrlp, ctrlp->
callback_data);
}
static void
! storeSwapInFileOpened(void *data, int fd, int errcode)
{
swapin_ctrl_t *ctrlp = (swapin_ctrl_t *) data;
StoreEntry *e = ctrlp->e;
MemObject *mem = e->mem_obj;
+ struct stat sb;
+
+ if(fd == -2 && errcode == -2) {
+ xfree(ctrlp->path);
+ xfree(ctrlp);
+ return;
+ }
+
assert(mem != NULL);
assert(e->mem_status == NOT_IN_MEMORY);
assert(e->swap_status == SWAPOUT_WRITING || e->swap_status ==
SWAPOUT_DONE);
+ if (e->swap_status == SWAPOUT_DONE && (fd>=0) && fstat(fd, &sb) == 0)
+ if(sb.st_size == 0 || sb.st_size != e->object_len) {
+ debug(20,0) ("storeSwapInFileOpened: %s: Size mismatch: %d(fstat) !=
%d(object)\n", ctrlp->path, sb.st_size, e->object_len);
+ file_close(fd);
+ fd = -1;
+ }
if (fd < 0) {
! debug(20, 0) ("storeSwapInFileOpened: Failed for '%s'\n", mem->url);
/* Invoke a store abort that should free the memory object */
(ctrlp->callback) (-1, ctrlp->callback_data);
xfree(ctrlp->path);
xfree(ctrlp);
return;
}
+
debug(20, 5) ("storeSwapInStart: initialized swap file '%s' for '%s'\n",
ctrlp->path, mem->url);
(ctrlp->callback) (fd, ctrlp->callback_data);
***************
*** 1113,1137 ****
RB->invalid++;
continue;
}
- storeSwapFullPath(sfileno, swapfile);
if (x != 9) {
RB->invalid++;
continue;
}
! if (sfileno < 0) {
! RB->invalid++;
continue;
}
if (EBIT_TEST(scan7, KEY_PRIVATE)) {
RB->badflags++;
continue;
}
sfileno = storeDirProperFileno(d->dirn, sfileno);
- timestamp = (time_t) scan1;
- lastref = (time_t) scan2;
- expires = (time_t) scan3;
- lastmod = (time_t) scan4;
- size = (off_t) scan5;
key = storeKeyScan(keytext);
if (key == NULL) {
--- 1169,1204 ----
RB->invalid++;
continue;
}
if (x != 9) {
RB->invalid++;
continue;
}
! timestamp = (time_t) scan1;
! lastref = (time_t) scan2;
! expires = (time_t) scan3;
! lastmod = (time_t) scan4;
! size = (off_t) scan5;
!
! if (size < 0) {
! if((key = storeKeyScan(keytext)) == NULL)
! continue;
! if((e = storeGet(key)) == NULL)
! continue;
! if(e->lastref > lastref)
! continue;
! debug(20, 3) ("storeRebuildFromDisk: Cancelling: '%s'\n", keytext);
! storeRelease(e);
! RB->objcount--;
! RB->cancelcount++;
continue;
}
+
+ storeSwapFullPath(sfileno, swapfile);
if (EBIT_TEST(scan7, KEY_PRIVATE)) {
RB->badflags++;
continue;
}
sfileno = storeDirProperFileno(d->dirn, sfileno);
key = storeKeyScan(keytext);
if (key == NULL) {
***************
*** 1174,1180 ****
/* We'll assume the existing entry is valid, probably because
* were in a slow rebuild and the the swap file number got taken
* and the validation procedure hasn't run. */
! assert(RB->need_to_validate);
RB->clashcount++;
continue;
} else if (e) {
--- 1241,1247 ----
/* We'll assume the existing entry is valid, probably because
* were in a slow rebuild and the the swap file number got taken
* and the validation procedure hasn't run. */
! /* assert(RB->need_to_validate); */
RB->clashcount++;
continue;
} else if (e) {
***************
*** 1211,1273 ****
static void
storeCleanup(void *datanotused)
{
- static storeCleanList *list = NULL;
- storeCleanList *curr;
static int bucketnum = -1;
static int validnum = 0;
StoreEntry *e;
hash_link *link_ptr = NULL;
! if (list == NULL) {
! if (++bucketnum >= store_hash_buckets) {
! debug(20, 1) (" Completed Validation Procedure\n");
! debug(20, 1) (" Validated %d Entries\n", validnum);
! debug(20, 1) (" store_swap_size = %dk\n", store_swap_size);
! store_rebuilding = 0;
! return;
! }
! link_ptr = hash_get_bucket(store_table, bucketnum);
! for (; link_ptr; link_ptr = link_ptr->next) {
! e = (StoreEntry *) link_ptr;
! if (EBIT_TEST(e->flag, ENTRY_VALIDATED))
! continue;
! if (EBIT_TEST(e->flag, RELEASE_REQUEST))
! continue;
! curr = xcalloc(1, sizeof(storeCleanList));
! curr->key = storeKeyDup(e->key);
! curr->next = list;
! list = curr;
! }
! }
! if (list == NULL) {
! eventAdd("storeCleanup", storeCleanup, NULL, 0);
! return;
! }
! curr = list;
! list = list->next;
! e = (StoreEntry *) hash_lookup(store_table, curr->key);
! if (e && !EBIT_TEST(e->flag, ENTRY_VALIDATED)) {
! storeLockObject(e);
! storeValidate(e, storeCleanupComplete, e);
! if ((++validnum & 0xFFF) == 0)
debug(20, 1) (" %7d Entries Validated so far.\n", validnum);
- assert(validnum <= meta_data.store_entries);
}
- storeKeyFree(curr->key);
- xfree(curr);
eventAdd("storeCleanup", storeCleanup, NULL, 0);
}
static void
! storeCleanupComplete(void *data)
{
StoreEntry *e = data;
storeUnlockObject(e);
if (!EBIT_TEST(e->flag, ENTRY_VALIDATED))
storeRelease(e);
}
static void
! storeValidate(StoreEntry * e, VCB callback, void *callback_data)
{
valid_ctrl_t *ctrlp;
char *path;
--- 1278,1327 ----
static void
storeCleanup(void *datanotused)
{
static int bucketnum = -1;
static int validnum = 0;
StoreEntry *e;
hash_link *link_ptr = NULL;
!
! if (++bucketnum >= store_hash_buckets) {
! debug(20, 1) (" Completed Validation Procedure\n");
! debug(20, 1) (" Validated %d Entries\n", validnum);
! debug(20, 1) (" store_swap_size = %dk\n", store_swap_size);
! store_rebuilding = 0;
! return;
! }
! link_ptr = hash_get_bucket(store_table, bucketnum);
! for (; link_ptr; link_ptr = link_ptr->next) {
! e = (StoreEntry *) link_ptr;
! if (EBIT_TEST(e->flag, ENTRY_VALIDATED))
! continue;
! if (EBIT_TEST(e->flag, RELEASE_REQUEST))
! continue;
! EBIT_SET(e->flag, ENTRY_VALIDATED);
! /* Only set the file bit if we know its a valid entry */
! /* otherwise, set it in the validation procedure */
! storeDirUpdateSwapSize(e->swap_file_number, e->object_len, 1);
! if ((++validnum & 0xFFFF) == 0)
debug(20, 1) (" %7d Entries Validated so far.\n", validnum);
}
eventAdd("storeCleanup", storeCleanup, NULL, 0);
}
static void
! storeCleanupComplete(void *data, int retcode, int errcode)
{
StoreEntry *e = data;
storeUnlockObject(e);
+
+ outvalid--;
+ if(retcode == -2 && errcode == -2)
+ return;
if (!EBIT_TEST(e->flag, ENTRY_VALIDATED))
storeRelease(e);
}
static void
! storeValidate(StoreEntry * e, VCB callback, void *callback_data, void *tag)
{
valid_ctrl_t *ctrlp;
char *path;
***************
*** 1278,1284 ****
assert(!EBIT_TEST(e->flag, ENTRY_VALIDATED));
if (e->swap_file_number < 0) {
EBIT_CLR(e->flag, ENTRY_VALIDATED);
! callback(callback_data);
return;
}
path = storeSwapFullPath(e->swap_file_number, NULL);
--- 1332,1338 ----
assert(!EBIT_TEST(e->flag, ENTRY_VALIDATED));
if (e->swap_file_number < 0) {
EBIT_CLR(e->flag, ENTRY_VALIDATED);
! callback(callback_data, 0, 0);
return;
}
path = storeSwapFullPath(e->swap_file_number, NULL);
***************
*** 1289,1295 ****
ctrlp->callback = callback;
ctrlp->callback_data = callback_data;
#if USE_ASYNC_IO
! aioStat(path, sb, storeValidateComplete, ctrlp);
#else
/* When evaluating the actual arguments in a function call, the order
* in which the arguments and the function expression are evaluated is
--- 1343,1349 ----
ctrlp->callback = callback;
ctrlp->callback_data = callback_data;
#if USE_ASYNC_IO
! aioStat(path, sb, storeValidateComplete, ctrlp, tag);
#else
/* When evaluating the actual arguments in a function call, the order
* in which the arguments and the function expression are evaluated is
***************
*** 1307,1312 ****
--- 1361,1373 ----
struct stat *sb = ctrlp->sb;
StoreEntry *e = ctrlp->e;
char *path;
+
+ if(retcode == -2 && errcode == -2) {
+ xfree(sb);
+ xfree(ctrlp);
+ ctrlp->callback(ctrlp->callback_data, retcode, errcode);
+ return;
+ }
if (retcode < 0 && errcode == EWOULDBLOCK) {
path = storeSwapFullPath(e->swap_file_number, NULL);
retcode = stat(path, sb);
***************
*** 1315,1325 ****
EBIT_CLR(e->flag, ENTRY_VALIDATED);
} else {
EBIT_SET(e->flag, ENTRY_VALIDATED);
- storeDirMapBitSet(e->swap_file_number);
storeDirUpdateSwapSize(e->swap_file_number, e->object_len, 1);
}
errno = errcode;
! ctrlp->callback(ctrlp->callback_data);
xfree(sb);
xfree(ctrlp);
}
--- 1376,1385 ----
EBIT_CLR(e->flag, ENTRY_VALIDATED);
} else {
EBIT_SET(e->flag, ENTRY_VALIDATED);
storeDirUpdateSwapSize(e->swap_file_number, e->object_len, 1);
}
errno = errcode;
! ctrlp->callback(ctrlp->callback_data, retcode, errcode);
xfree(sb);
xfree(ctrlp);
}
***************
*** 1338,1354 ****
debug(20, 1) (" %7d With invalid flags.\n", data->badflags);
debug(20, 1) (" %7d Objects loaded.\n", data->objcount);
debug(20, 1) (" %7d Objects expired.\n", data->expcount);
debug(20, 1) (" %7d Duplicate URLs purged.\n", data->dupcount);
debug(20, 1) (" %7d Swapfile clashes avoided.\n", data->clashcount);
debug(20, 1) (" Took %d seconds (%6.1lf objects/sec).\n",
r > 0 ? r : 0, (double) data->objcount / (r > 0 ? r : 1));
! if (data->need_to_validate && data->linecount) {
! debug(20, 1) ("Beginning Validation Procedure\n");
! eventAdd("storeCleanup", storeCleanup, NULL, 0);
! } else {
! debug(20, 1) (" store_swap_size = %dk\n", store_swap_size);
! store_rebuilding = 0;
! }
safe_free(data->line_in);
safe_free(data);
}
--- 1398,1410 ----
debug(20, 1) (" %7d With invalid flags.\n", data->badflags);
debug(20, 1) (" %7d Objects loaded.\n", data->objcount);
debug(20, 1) (" %7d Objects expired.\n", data->expcount);
+ debug(20, 1) (" %7d Objects cancelled.\n", data->cancelcount);
debug(20, 1) (" %7d Duplicate URLs purged.\n", data->dupcount);
debug(20, 1) (" %7d Swapfile clashes avoided.\n", data->clashcount);
debug(20, 1) (" Took %d seconds (%6.1lf objects/sec).\n",
r > 0 ? r : 0, (double) data->objcount / (r > 0 ? r : 1));
! debug(20, 1) ("Beginning Validation Procedure\n");
! eventAdd("storeCleanup", storeCleanup, NULL, 0);
safe_free(data->line_in);
safe_free(data);
}
***************
*** 1465,1473 ****
InvokeHandlers(e);
/* Do we need to close the swapout file? */
/* Not if we never started swapping out */
if (e->swap_file_number == -1)
return;
! /* not if a disk write is queued, the handler will close up */
if (mem->swapout.queue_offset > mem->swapout.done_offset)
return;
/* we do */
--- 1521,1536 ----
InvokeHandlers(e);
/* Do we need to close the swapout file? */
/* Not if we never started swapping out */
+ /* But we may need to cancel an open/stat in progress if using ASYNC */
+ #if USE_ASYNC_IO
+ aioCancel(-1, e);
+ #endif
if (e->swap_file_number == -1)
return;
! /* Need to cancel any pending ASYNC writes right now */
! if(mem->swapout.fd >= 0)
! aioCancel(mem->swapout.fd, NULL);
! /* but dont close if a disk write is queued, the handler will close up */
if (mem->swapout.queue_offset > mem->swapout.done_offset)
return;
/* we do */
***************
*** 1546,1562 ****
int expired = 0;
int max_scan;
int max_remove;
static time_t last_warn_time = 0;
eventAdd("storeMaintainSwapSpace", storeMaintainSwapSpace, NULL, 1);
/* We can't delete objects while rebuilding swap */
if (store_rebuilding)
return;
! if (store_swap_size < store_swap_high) {
! max_scan = 100;
! max_remove = 10;
} else {
! max_scan = 500;
! max_remove = 50;
}
debug(20, 3) ("storeMaintainSwapSpace\n");
for (m = all_list.tail; m; m = prev) {
--- 1609,1632 ----
int expired = 0;
int max_scan;
int max_remove;
+ int bigclean = 0;
+ int level = 3;
static time_t last_warn_time = 0;
eventAdd("storeMaintainSwapSpace", storeMaintainSwapSpace, NULL, 1);
/* We can't delete objects while rebuilding swap */
if (store_rebuilding)
return;
!
! if (store_swap_size > store_swap_high)
! bigclean = 1;
! if (store_swap_size > Config.Swap.maxSize)
! bigclean = 1;
!
! if (bigclean) {
! max_scan = 2500;
! max_remove = 250;
} else {
! return;
}
debug(20, 3) ("storeMaintainSwapSpace\n");
for (m = all_list.tail; m; m = prev) {
***************
*** 1564,1583 ****
e = m->data;
if (storeEntryLocked(e)) {
locked++;
! } else if (storeCheckExpired(e, 1)) {
expired++;
storeRelease(e);
}
if (expired > max_remove)
break;
if (++scanned > max_scan)
break;
}
! debug(20, 3) ("storeMaintainSwapSpace stats:\n");
! debug(20, 3) (" %6d objects\n", meta_data.store_entries);
! debug(20, 3) (" %6d were scanned\n", scanned);
! debug(20, 3) (" %6d were locked\n", locked);
! debug(20, 3) (" %6d were expired\n", expired);
if (store_swap_size < Config.Swap.maxSize)
return;
if (squid_curtime - last_warn_time < 10)
--- 1634,1661 ----
e = m->data;
if (storeEntryLocked(e)) {
locked++;
! continue;
! } else if (bigclean) {
expired++;
storeRelease(e);
+ } else {
+ if (storeCheckExpired(e, 1)) {
+ expired++;
+ storeRelease(e);
+ }
}
if (expired > max_remove)
break;
if (++scanned > max_scan)
break;
}
! if(bigclean)
! level = 1;
! debug(20, level) ("storeMaintainSwapSpace stats:\n");
! debug(20, level) (" %6d objects\n", meta_data.store_entries);
! debug(20, level) (" %6d were scanned\n", scanned);
! debug(20, level) (" %6d were locked\n", locked);
! debug(20, level) (" %6d were expired\n", expired);
if (store_swap_size < Config.Swap.maxSize)
return;
if (squid_curtime - last_warn_time < 10)
***************
*** 1602,1622 ****
storeReleaseRequest(e);
return 0;
}
if (store_rebuilding) {
debug(20, 2) ("storeRelease: Delaying release until store is rebuilt:
'%s'\n",
storeUrl(e));
storeExpireNow(e);
storeSetPrivateKey(e);
EBIT_SET(e->flag, RELEASE_REQUEST);
return 0;
}
storeLog(STORE_LOG_RELEASE, e);
if (e->swap_file_number > -1) {
! if (EBIT_TEST(e->flag, ENTRY_VALIDATED))
! storePutUnusedFileno(e->swap_file_number);
if (e->swap_status == SWAPOUT_DONE)
storeDirUpdateSwapSize(e->swap_file_number, e->object_len, -1);
! e->swap_file_number = -1;
}
storeSetMemStatus(e, NOT_IN_MEMORY);
destroy_StoreEntry(e);
--- 1680,1716 ----
storeReleaseRequest(e);
return 0;
}
+ #if USE_ASYNC_IO
+ aioCancel(-1, e); /* Make sure all forgotten async ops are cancelled */
+ #else
if (store_rebuilding) {
debug(20, 2) ("storeRelease: Delaying release until store is rebuilt:
'%s'\n",
storeUrl(e));
storeExpireNow(e);
storeSetPrivateKey(e);
EBIT_SET(e->flag, RELEASE_REQUEST);
+ e->object_len = -(e->object_len);
+ storeDirSwapLog(e);
+ e->object_len = -(e->object_len);
return 0;
}
+ #endif
storeLog(STORE_LOG_RELEASE, e);
if (e->swap_file_number > -1) {
! #if MONOTONIC_STORE
! #if USE_ASYNC_IO
! safeunlink(storeSwapFullPath(e->swap_file_number, NULL), 1);
! #else
! unlinkdUnlink(storeSwapFullPath(e->swap_file_number, NULL));
! #endif
! #else
! storePutUnusedFileno(e);
! #endif
if (e->swap_status == SWAPOUT_DONE)
storeDirUpdateSwapSize(e->swap_file_number, e->object_len, -1);
! e->object_len = -(e->object_len);
! storeDirSwapLog(e);
! e->object_len = -(e->object_len);
}
storeSetMemStatus(e, NOT_IN_MEMORY);
destroy_StoreEntry(e);
***************
*** 1629,1634 ****
--- 1723,1730 ----
{
if (e->lock_count)
return 1;
+ if (e->swap_status == SWAPOUT_OPENING)
+ return 1;
if (e->swap_status == SWAPOUT_WRITING)
return 1;
if (e->store_status == STORE_PENDING)
***************
*** 1680,1685 ****
--- 1776,1782 ----
sc->seen_offset = 0;
sc->copy_offset = 0;
sc->swapin_fd = -1;
+ sc->disk_op_in_progress = 0;
sc->mem = mem;
if (e->store_status == STORE_PENDING && mem->swapout.fd == -1)
sc->type = STORE_MEM_CLIENT;
***************
*** 1701,1707 ****
{
store_client *sc;
static int recurse_detect = 0;
! /*assert(e->store_status != STORE_ABORTED); */
assert(recurse_detect < 3); /* could == 1 for IMS not modified's */
debug(20, 3) ("storeClientCopy: %s, seen %d, want %d, size %d, cb %p,
cbdata %p\n",
storeKeyText(e->key),
--- 1798,1804 ----
{
store_client *sc;
static int recurse_detect = 0;
! assert(e->store_status != STORE_ABORTED);
assert(recurse_detect < 3); /* could == 1 for IMS not modified's */
debug(20, 3) ("storeClientCopy: %s, seen %d, want %d, size %d, cb %p,
cbdata %p\n",
storeKeyText(e->key),
***************
*** 1734,1743 ****
--- 1831,1858 ----
debug(20, 3) ("storeClientCopy2: %s\n", storeKeyText(e->key));
assert(callback != NULL);
if (e->store_status == STORE_ABORTED) {
+ #if USE_ASYNC_IO
+ if(sc->disk_op_in_progress == 1) {
+ if(sc->swapin_fd >= 0)
+ aioCancel(sc->swapin_fd, NULL);
+ else
+ aioCancel(-1, sc);
+ sc->disk_op_in_progress = 0;
+ }
+ #endif
sc->callback = NULL;
callback(sc->callback_data, sc->copy_buf, 0);
} else if (e->store_status == STORE_OK && sc->copy_offset == e->
object_len) {
/* There is no more to send! */
+ #if USE_ASYNC_IO
+ if(sc->disk_op_in_progress == 1) {
+ if(sc->swapin_fd >= 0)
+ aioCancel(sc->swapin_fd, NULL);
+ else
+ aioCancel(-1, sc);
+ sc->disk_op_in_progress = 0;
+ }
+ #endif
sc->callback = NULL;
callback(sc->callback_data, sc->copy_buf, 0);
} else if (e->store_status == STORE_PENDING && sc->seen_offset == mem->
inmem_hi) {
***************
*** 1747,1752 ****
--- 1862,1876 ----
/* What the client wants is in memory */
debug(20, 3) ("storeClientCopy2: Copying from memory\n");
sz = memCopy(mem->data, sc->copy_offset, sc->copy_buf, sc->copy_size);
+ #if USE_ASYNC_IO
+ if(sc->disk_op_in_progress == 1) {
+ if(sc->swapin_fd >= 0)
+ aioCancel(sc->swapin_fd, NULL);
+ else
+ aioCancel(-1, sc);
+ sc->disk_op_in_progress = 0;
+ }
+ #endif
sc->callback = NULL;
callback(sc->callback_data, sc->copy_buf, sz);
} else if (sc->swapin_fd < 0) {
***************
*** 1754,1765 ****
assert(sc->type == STORE_DISK_CLIENT);
/* gotta open the swapin file */
/* assert(sc->copy_offset == 0); */
! storeSwapInStart(e, storeClientCopyFileOpened, sc);
} else {
debug(20, 3) ("storeClientCopy: reading from disk FD %d\n",
sc->swapin_fd);
assert(sc->type == STORE_DISK_CLIENT);
! storeClientCopyFileRead(sc);
}
--loopdetect;
}
--- 1878,1899 ----
assert(sc->type == STORE_DISK_CLIENT);
/* gotta open the swapin file */
/* assert(sc->copy_offset == 0); */
! if(sc->disk_op_in_progress == 0) {
! sc->disk_op_in_progress = 1;
! storeSwapInStart(e, storeClientCopyFileOpened, sc);
! } else {
! debug(20, 2) ("storeClientCopy2: Averted multiple fd operation\n");
! }
} else {
debug(20, 3) ("storeClientCopy: reading from disk FD %d\n",
sc->swapin_fd);
assert(sc->type == STORE_DISK_CLIENT);
! if(sc->disk_op_in_progress == 0) {
! sc->disk_op_in_progress = 1;
! storeClientCopyFileRead(sc);
! } else {
! debug(20, 2) ("storeClientCopy2: Averted multiple fd operation\n");
! }
}
--loopdetect;
}
***************
*** 1771,1776 ****
--- 1905,1911 ----
STCB *callback = sc->callback;
if (fd < 0) {
debug(20, 3) ("storeClientCopyFileOpened: failed\n");
+ sc->disk_op_in_progress = 0;
sc->callback = NULL;
callback(sc->callback_data, sc->copy_buf, -1);
return;
***************
*** 1797,1802 ****
--- 1932,1939 ----
store_client *sc = data;
MemObject *mem = sc->mem;
STCB *callback = sc->callback;
+ assert(sc->disk_op_in_progress != 0);
+ sc->disk_op_in_progress = 0;
assert(sc->callback != NULL);
debug(20, 3) ("storeClientCopyHandleRead: FD %d, len %d\n", fd, len);
if (sc->copy_offset == 0 && len > 0 && mem != NULL)
***************
*** 1896,1902 ****
if (strcmp((fname = Config.Log.store), "none") == 0)
storelog_fd = -1;
else
! storelog_fd = file_open(fname, O_WRONLY | O_CREAT, NULL, NULL);
if (storelog_fd < 0)
debug(20, 1) ("Store logging disabled\n");
if (storeVerifyCacheDirs() < 0) {
--- 2033,2039 ----
if (strcmp((fname = Config.Log.store), "none") == 0)
storelog_fd = -1;
else
! storelog_fd = file_open(fname, O_WRONLY | O_CREAT, NULL, NULL, NULL);
if (storelog_fd < 0)
debug(20, 1) ("Store logging disabled\n");
if (storeVerifyCacheDirs() < 0) {
***************
*** 1979,1989 ****
cur[dirn] = xstrdup(storeDirSwapLogFile(dirn, NULL));
new[dirn] = xstrdup(storeDirSwapLogFile(dirn, ".clean"));
cln[dirn] = xstrdup(storeDirSwapLogFile(dirn, ".last-clean"));
! safeunlink(new[dirn], 1);
! safeunlink(cln[dirn], 1);
fd[dirn] = file_open(new[dirn],
O_WRONLY | O_CREAT | O_TRUNC,
NULL,
NULL);
if (fd[dirn] < 0) {
debug(50, 0) ("storeWriteCleanLogs: %s: %s\n", new[dirn], xstrerror());
--- 2116,2127 ----
cur[dirn] = xstrdup(storeDirSwapLogFile(dirn, NULL));
new[dirn] = xstrdup(storeDirSwapLogFile(dirn, ".clean"));
cln[dirn] = xstrdup(storeDirSwapLogFile(dirn, ".last-clean"));
! unlink(new[dirn]);
! unlink(cln[dirn]);
fd[dirn] = file_open(new[dirn],
O_WRONLY | O_CREAT | O_TRUNC,
NULL,
+ NULL,
NULL);
if (fd[dirn] < 0) {
debug(50, 0) ("storeWriteCleanLogs: %s: %s\n", new[dirn], xstrerror());
***************
*** 2001,2007 ****
outbufs[dirn] = xcalloc(Config.cacheSwap.n_configured, CLEAN_BUF_SZ);
outbuflens[dirn] = 0;
}
! for (m = all_list.head; m; m = m->next) {
e = m->data;
if (e->swap_file_number < 0)
continue;
--- 2139,2145 ----
outbufs[dirn] = xcalloc(Config.cacheSwap.n_configured, CLEAN_BUF_SZ);
outbuflens[dirn] = 0;
}
! for (m = all_list.tail; m; m = m->prev) {
e = m->data;
if (e->swap_file_number < 0)
continue;
***************
*** 2035,2041 ****
debug(20, 0) ("storeWriteCleanLogs: Current swap logfile not replaced.\n");
file_close(fd[dirn]);
fd[dirn] = -1;
! safeunlink(cln[dirn], 0);
continue;
}
outbuflens[dirn] = 0;
--- 2173,2179 ----
debug(20, 0) ("storeWriteCleanLogs: Current swap logfile not replaced.\n");
file_close(fd[dirn]);
fd[dirn] = -1;
! unlink(cln[dirn]);
continue;
}
outbuflens[dirn] = 0;
***************
*** 2056,2062 ****
debug(20, 0) ("storeWriteCleanLogs: Current swap logfile not replaced.\n");
file_close(fd[dirn]);
fd[dirn] = -1;
! safeunlink(cln[dirn], 0);
continue;
}
}
--- 2194,2200 ----
debug(20, 0) ("storeWriteCleanLogs: Current swap logfile not replaced.\n");
file_close(fd[dirn]);
fd[dirn] = -1;
! unlink(cln[dirn]);
continue;
}
}
***************
*** 2069,2076 ****
file_close(fd[dirn]);
fd[dirn] = -1;
if (rename(new[dirn], cur[dirn]) < 0) {
! debug(50, 0) ("storeWriteCleanLogs: rename failed: %s\n",
! xstrerror());
}
}
storeDirCloseSwapLogs();
--- 2207,2214 ----
file_close(fd[dirn]);
fd[dirn] = -1;
if (rename(new[dirn], cur[dirn]) < 0) {
! debug(50, 0) ("storeWriteCleanLogs: rename failed: %s, %s -> %s\n",
! xstrerror(), new[dirn], cur[dirn]);
}
}
storeDirCloseSwapLogs();
***************
*** 2085,2091 ****
for (dirn = 0; dirn < Config.cacheSwap.n_configured; dirn++) {
if (!store_rebuilding)
file_close(file_open(cln[dirn],
! O_WRONLY | O_CREAT | O_TRUNC, NULL, NULL));
safe_free(cur[dirn]);
safe_free(new[dirn]);
safe_free(cln[dirn]);
--- 2223,2229 ----
for (dirn = 0; dirn < Config.cacheSwap.n_configured; dirn++) {
if (!store_rebuilding)
file_close(file_open(cln[dirn],
! O_WRONLY | O_CREAT | O_TRUNC, NULL, NULL, NULL));
safe_free(cur[dirn]);
safe_free(new[dirn]);
safe_free(cln[dirn]);
***************
*** 2109,2115 ****
return 0;
for (sc = mem->clients; sc; sc = nx) {
nx = sc->next;
! if (sc->callback_data == NULL)
continue;
npend++;
}
--- 2247,2256 ----
return 0;
for (sc = mem->clients; sc; sc = nx) {
nx = sc->next;
! /* Changed from callback_data to just callback. There can be no use */
! /* for callback_data without a callback, and sc->callback we know */
! /* gets reset, but not necessarily sc->callback_data */
! if (sc->callback == NULL)
continue;
npend++;
}
***************
*** 2155,2161 ****
snprintf(to, MAXPATHLEN, "%s.%d", fname, 0);
rename(fname, to);
}
! storelog_fd = file_open(fname, O_WRONLY | O_CREAT, NULL, NULL);
if (storelog_fd < 0) {
debug(50, 0) ("storeRotateLog: %s: %s\n", fname, xstrerror());
debug(20, 1) ("Store logging disabled\n");
--- 2296,2302 ----
snprintf(to, MAXPATHLEN, "%s.%d", fname, 0);
rename(fname, to);
}
! storelog_fd = file_open(fname, O_WRONLY | O_CREAT, NULL, NULL, NULL);
if (storelog_fd < 0) {
debug(50, 0) ("storeRotateLog: %s: %s\n", fname, xstrerror());
debug(20, 1) ("Store logging disabled\n");
***************
*** 2288,2293 ****
--- 2429,2435 ----
#define FILENO_STACK_SIZE 128
static int fileno_stack[FILENO_STACK_SIZE];
+ #if !MONOTONIC_STORE
static int
storeGetUnusedFileno(void)
{
***************
*** 2301,2315 ****
}
static void
! storePutUnusedFileno(int fileno)
{
! assert(storeDirMapBitTest(fileno));
! storeDirMapBitReset(fileno);
if (fileno_stack_count < FILENO_STACK_SIZE)
! fileno_stack[fileno_stack_count++] = fileno;
else
! unlinkdUnlink(storeSwapFullPath(fileno, NULL));
}
void
storeRegisterAbort(StoreEntry * e, STABH * cb, void *data)
--- 2443,2474 ----
}
static void
! storePutUnusedFileno(StoreEntry *e)
{
! assert(storeDirMapBitTest(e->swap_file_number));
! storeDirMapBitReset(e->swap_file_number);
! /* If we're still rebuilding the swap state, then we need to avoid the */
! /* race condition where a new object gets pulled in, it expires, gets */
! /* its swapfileno added to the stack, and then that swapfileno gets */
! /* claimed by the rebuild. Must still remove the file though in any */
! /* event to avoid serving up the wrong data. This will leave us with */
! /* a URL pointing to no file at all, but that's okay since it'll fail */
! /* and get removed later anyway. */
! if(store_rebuilding) {
! if(EBIT_TEST(e->flag, ENTRY_VALIDATED))
! safeunlink(storeSwapFullPath(e->swap_file_number, NULL), 1);
! return;
! }
if (fileno_stack_count < FILENO_STACK_SIZE)
! fileno_stack[fileno_stack_count++] = e->swap_file_number;
else
! #if USE_ASYNC_IO
! safeunlink(storeSwapFullPath(e->swap_file_number, NULL), 1);
! #else
! unlinkdUnlink(storeSwapFullPath(e->swap_file_number, NULL));
! #endif
}
+ #endif
void
storeRegisterAbort(StoreEntry * e, STABH * cb, void *data)
***************
*** 2381,2386 ****
--- 2540,2549 ----
MemObject *mem = e->mem_obj;
if (mem->swapout.fd > -1)
file_close(mem->swapout.fd);
+ #if USE_ASYNC_IO
+ else
+ aioCancel(-1, e); /* Make doubly certain pending ops are gone */
+ #endif
mem->swapout.fd = -1;
storeUnlockObject(e);
}
diff -cr squid-1.2.beta11/src/store_dir.c squid-1.2.slf/src/store_dir.c
*** squid-1.2.beta11/src/store_dir.c Tue Jan 6 18:11:56 1998
--- squid-1.2.slf/src/store_dir.c Thu Jan 29 18:08:38 1998
***************
*** 1,6 ****
/*
! * $Id: store_dir.c,v 1.39 1998/01/06 07:11:56 wessels Exp $
*
* DEBUG: section 47 Store Directory Routines
* AUTHOR: Duane Wessels
--- 1,6 ----
/*
! * $Id: store_dir.c,v 1.4 1998/01/29 01:24:05 slf Exp slf $
*
* DEBUG: section 47 Store Directory Routines
* AUTHOR: Duane Wessels
***************
*** 52,62 ****
--- 52,70 ----
if (!fullpath)
fullpath = fullfilename;
fullpath[0] = '\0';
+ #if MONOTONIC_STORE
+ snprintf(fullpath, SQUID_MAXPATHLEN, "%s/%02X/%02X/%08X",
+ Config.cacheSwap.swapDirs[dirn].path,
+ ((filn / Config.cacheSwap.swapDirs[dirn].l2) / Config.cacheSwap.swapDirs[dir
n].l2) % Config.cacheSwap.swapDirs[dirn].l1,
+ (filn / Config.cacheSwap.swapDirs[dirn].l2) % Config.cacheSwap.swapDirs[dirn
].l2,
+ filn);
+ #else
snprintf(fullpath, SQUID_MAXPATHLEN, "%s/%02X/%02X/%08X",
Config.cacheSwap.swapDirs[dirn].path,
filn % Config.cacheSwap.swapDirs[dirn].l1,
filn / Config.cacheSwap.swapDirs[dirn].l1 % Config.cacheSwap.swapDirs[dirn].
l2,
filn);
+ #endif
return fullpath;
}
***************
*** 83,92 ****
--- 91,107 ----
if (!fullpath)
fullpath = fullfilename;
fullpath[0] = '\0';
+ #if MONOTONIC_STORE
snprintf(fullpath, SQUID_MAXPATHLEN, "%s/%02X/%02X",
Config.cacheSwap.swapDirs[dirn].path,
+ ((filn / Config.cacheSwap.swapDirs[dirn].l2) / Config.cacheSwap.swapDirs[dir
n].l2) % Config.cacheSwap.swapDirs[dirn].l1,
+ (filn / Config.cacheSwap.swapDirs[dirn].l2) % Config.cacheSwap.swapDirs[dirn
].l2);
+ #else
+ snprintf(fullpath, SQUID_MAXPATHLEN, "%s/%02X/%02X/%08X",
+ Config.cacheSwap.swapDirs[dirn].path,
filn % Config.cacheSwap.swapDirs[dirn].l1,
filn / Config.cacheSwap.swapDirs[dirn].l1 % Config.cacheSwap.swapDirs[dirn].
l2);
+ #endif
return fullpath;
}
***************
*** 212,218 ****
--- 227,235 ----
int dirn = fn >> SWAP_DIR_SHIFT;
int filn = fn & SWAP_FILE_MASK;
file_map_bit_set(Config.cacheSwap.swapDirs[dirn].map, filn);
+ #if !MONOTONIC_STORE
Config.cacheSwap.swapDirs[dirn].suggest++;
+ #endif
}
void
***************
*** 221,228 ****
--- 238,247 ----
int dirn = fn >> SWAP_DIR_SHIFT;
int filn = fn & SWAP_FILE_MASK;
file_map_bit_reset(Config.cacheSwap.swapDirs[dirn].map, filn);
+ #if !MONOTONIC_STORE
if (fn < Config.cacheSwap.swapDirs[dirn].suggest)
Config.cacheSwap.swapDirs[dirn].suggest = fn;
+ #endif
}
int
***************
*** 231,236 ****
--- 250,258 ----
int dirn = storeMostFreeSwapDir();
SwapDir *SD = &Config.cacheSwap.swapDirs[dirn];
int filn = file_map_allocate(SD->map, SD->suggest);
+ #if MONOTONIC_STORE
+ SD->suggest = filn + 1;
+ #endif
return (dirn << SWAP_DIR_SHIFT) | (filn & SWAP_FILE_MASK);
}
***************
*** 258,269 ****
{
LOCAL_ARRAY(char, logmsg, MAX_URL << 1);
int dirn;
assert(e->swap_file_number >= 0);
dirn = e->swap_file_number >> SWAP_DIR_SHIFT;
assert(dirn < Config.cacheSwap.n_configured);
assert(!EBIT_TEST(e->flag, KEY_PRIVATE));
/* Note this printf format appears in storeWriteCleanLog() too */
! snprintf(logmsg, MAX_URL << 1, "%08x %08x %08x %08x %08x %9d %6d %08x
%s\n",
(int) e->swap_file_number,
(int) e->timestamp,
(int) e->lastref,
--- 280,294 ----
{
LOCAL_ARRAY(char, logmsg, MAX_URL << 1);
int dirn;
+
assert(e->swap_file_number >= 0);
dirn = e->swap_file_number >> SWAP_DIR_SHIFT;
assert(dirn < Config.cacheSwap.n_configured);
+ #if !USE_ASYNC_IO
assert(!EBIT_TEST(e->flag, KEY_PRIVATE));
+ #endif
/* Note this printf format appears in storeWriteCleanLog() too */
! snprintf(logmsg, MAX_URL << 1, "%08x %08x %08x %08x %08x %d %6d %08x
%s\n",
(int) e->swap_file_number,
(int) e->timestamp,
(int) e->lastref,
***************
*** 310,316 ****
for (i = 0; i < Config.cacheSwap.n_configured; i++) {
SD = &Config.cacheSwap.swapDirs[i];
path = storeDirSwapLogFile(i, NULL);
! fd = file_open(path, O_WRONLY | O_CREAT, NULL, NULL);
if (fd < 0) {
debug(50, 1) ("%s: %s\n", path, xstrerror());
fatal("storeDirOpenSwapLogs: Failed to open swap log.");
--- 335,341 ----
for (i = 0; i < Config.cacheSwap.n_configured; i++) {
SD = &Config.cacheSwap.swapDirs[i];
path = storeDirSwapLogFile(i, NULL);
! fd = file_open(path, O_WRONLY | O_CREAT, NULL, NULL, NULL);
if (fd < 0) {
debug(50, 1) ("%s: %s\n", path, xstrerror());
fatal("storeDirOpenSwapLogs: Failed to open swap log.");
***************
*** 357,363 ****
if (SD->swaplog_fd >= 0)
file_close(SD->swaplog_fd);
/* open a write-only FD for the new log */
! fd = file_open(new_path, O_WRONLY | O_CREAT | O_TRUNC, NULL, NULL);
if (fd < 0) {
debug(50, 1) ("%s: %s\n", new_path, xstrerror());
fatal("storeDirOpenTmpSwapLog: Failed to open swap log.");
--- 382,388 ----
if (SD->swaplog_fd >= 0)
file_close(SD->swaplog_fd);
/* open a write-only FD for the new log */
! fd = file_open(new_path, O_WRONLY | O_CREAT | O_TRUNC, NULL, NULL, NULL);
if (fd < 0) {
debug(50, 1) ("%s: %s\n", new_path, xstrerror());
fatal("storeDirOpenTmpSwapLog: Failed to open swap log.");
***************
*** 395,401 ****
fatal("storeDirCloseTmpSwapLog: rename failed");
}
file_close(SD->swaplog_fd);
! fd = file_open(swaplog_path, O_WRONLY | O_CREAT, NULL, NULL);
if (fd < 0) {
debug(50, 1) ("%s: %s\n", swaplog_path, xstrerror());
fatal("storeDirCloseTmpSwapLog: Failed to open swap log.");
--- 420,426 ----
fatal("storeDirCloseTmpSwapLog: rename failed");
}
file_close(SD->swaplog_fd);
! fd = file_open(swaplog_path, O_WRONLY | O_CREAT, NULL, NULL, NULL);
if (fd < 0) {
debug(50, 1) ("%s: %s\n", swaplog_path, xstrerror());
fatal("storeDirCloseTmpSwapLog: Failed to open swap log.");
diff -cr squid-1.2.beta11/src/structs.h squid-1.2.slf/src/structs.h
*** squid-1.2.beta11/src/structs.h Tue Jan 6 16:12:10 1998
--- squid-1.2.slf/src/structs.h Thu Jan 8 16:13:17 1998
***************
*** 826,831 ****
--- 826,832 ----
void *callback_data;
MemObject *mem; /* ptr to the parent structure, argh! */
int swapin_fd;
+ int disk_op_in_progress;
struct _store_client *next;
};
diff -cr squid-1.2.beta11/src/tools.c squid-1.2.slf/src/tools.c
*** squid-1.2.beta11/src/tools.c Sun Jan 4 16:43:48 1998
--- squid-1.2.slf/src/tools.c Fri Jan 30 12:10:29 1998
***************
*** 1,6 ****
/*
! * $Id: tools.c,v 1.141 1998/01/04 05:43:48 wessels Exp $
*
* DEBUG: section 21 Misc Functions
* AUTHOR: Harvest Derived
--- 1,6 ----
/*
! * $Id: tools.c,v 1.2 1998/01/09 03:39:00 slf Exp slf $
*
* DEBUG: section 21 Misc Functions
* AUTHOR: Harvest Derived
***************
*** 384,390 ****
--- 384,392 ----
leave_suid();
}
releaseServerSockets();
+ #if !USE_ASYNC_IO
unlinkdClose();
+ #endif
storeWriteCleanLogs(0);
PrintRusage();
dumpMallocStats();
***************
*** 629,635 ****
return;
enter_suid();
old_umask = umask(022);
! fd = file_open(f, O_WRONLY | O_CREAT | O_TRUNC, NULL, NULL);
umask(old_umask);
leave_suid();
if (fd < 0) {
--- 631,637 ----
return;
enter_suid();
old_umask = umask(022);
! fd = file_open(f, O_WRONLY | O_CREAT | O_TRUNC, NULL, NULL, NULL);
umask(old_umask);
leave_suid();
if (fd < 0) {
diff -cr squid-1.2.beta11/src/typedefs.h squid-1.2.slf/src/typedefs.h
*** squid-1.2.beta11/src/typedefs.h Sat Dec 6 16:17:04 1997
--- squid-1.2.slf/src/typedefs.h Tue Jan 13 15:11:22 1998
***************
*** 80,86 ****
typedef void CWCB(int fd, char *, size_t size, int flag, void *data);
typedef void CNCB(int fd, int status, void *);
typedef void FREE(void *);
! typedef void FOCB(void *, int fd);
typedef void EVH(void *);
typedef void PF(int, void *);
typedef void DRCB(int fd, const char *buf, int size, int errflag, void
*data);
--- 80,86 ----
typedef void CWCB(int fd, char *, size_t size, int flag, void *data);
typedef void CNCB(int fd, int status, void *);
typedef void FREE(void *);
! typedef void FOCB(void *, int fd, int errcode);
typedef void EVH(void *);
typedef void PF(int, void *);
typedef void DRCB(int fd, const char *buf, int size, int errflag, void
*data);
diff -cr squid-1.2.beta11/src/unlinkd.c squid-1.2.slf/src/unlinkd.c
*** squid-1.2.beta11/src/unlinkd.c Thu Jan 1 16:57:17 1998
--- squid-1.2.slf/src/unlinkd.c Fri Jan 30 12:11:02 1998
***************
*** 77,82 ****
--- 77,84 ----
#include "squid.h"
+ #if !USE_ASYNC_IO
+
static int unlinkd_fd = -1;
static int unlinkdCreate(void);
***************
*** 200,203 ****
--- 202,206 ----
debug(43, 0) ("Unlinkd pipe opened on FD %d\n", unlinkd_fd);
}
+ #endif /* !USE_ASYNC_IO */
#endif /* ndef UNLINK_DAEMON */
diff -cr squid-1.2.beta11/src/useragent.c squid-1.2.slf/src/useragent.c
*** squid-1.2.beta11/src/useragent.c Fri Oct 17 10:00:50 1997
--- squid-1.2.slf/src/useragent.c Fri Jan 9 14:38:53 1998
***************
*** 1,6 ****
/*
! * $Id: useragent.c,v 1.12 1997/10/17 00:00:50 wessels Exp $
*
* DEBUG: section 40 User-Agent logging
* AUTHOR: Joe Ramey <ramey@csc.ti.com>
--- 1,6 ----
/*
! * $Id: useragent.c,v 1.2 1998/01/09 03:38:37 slf Exp slf $
*
* DEBUG: section 40 User-Agent logging
* AUTHOR: Joe Ramey <ramey@csc.ti.com>
***************
*** 46,52 ****
cache_useragent_log = NULL;
}
if (fname && strcmp(fname, "none") != 0) {
! log_fd = file_open(fname, O_WRONLY | O_CREAT | O_APPEND, NULL, NULL);
if (log_fd < 0) {
debug(50, 0) ("useragentOpenLog: %s: %s\n", fname, xstrerror());
} else if ((cache_useragent_log = fdopen(log_fd, "a")) == NULL) {
--- 46,52 ----
cache_useragent_log = NULL;
}
if (fname && strcmp(fname, "none") != 0) {
! log_fd = file_open(fname, O_WRONLY | O_CREAT | O_APPEND, NULL, NULL, NULL);
if (log_fd < 0) {
debug(50, 0) ("useragentOpenLog: %s: %s\n", fname, xstrerror());
} else if ((cache_useragent_log = fdopen(log_fd, "a")) == NULL) {
--MimeMultipartBoundary--
Received on Tue Jul 29 2003 - 13:15:45 MDT
This archive was generated by hypermail pre-2.1.9 : Tue Dec 09 2003 - 16:11:34 MST