Dealing with "past" events

[Don Y]

Given:
N = "now"
R = task release time
D = task's deadline time
E = event time

The only real invariant is that R < N as code executing in a
task can't execute until the task has been released.

Specifically, there are no guaranteed relationship between
*most* of these times; N > E, N > D, D < R, etc. are all
possible in a generic system.

Though, in a nominal setting, R < N < E < D when the code tries
to initiate an event at a particular (future) time (E).

But, what happens when E < N -- i.e., when you try to schedule
an action (event) at a time that has already passed? And, does
E << N lead to a different interpretation/handling?

I've argued that the OS shouldn't special-case such activities.
If you request something to happen in the past, then the OS
should just act as if it has *just* happened, regardless as to
whether you were a microsecond "late" in issuing your request
or *years*! In particular, the OS shouldn't dismiss such request
unilaterally -- or, throw an error to alert the issuer to the
*apparent* inconsistency.

[George Neuner]

Post by Don Y
N = "now"
R = task release time
D = task's deadline time
E = event time
The only real invariant is that R < N as code executing in a
task can't execute until the task has been released.
Specifically, there are no guaranteed relationship between
*most* of these times; N > E, N > D, D < R, etc. are all
possible in a generic system.
Though, in a nominal setting, R < N < E < D when the code tries
to initiate an event at a particular (future) time (E).
But, what happens when E < N -- i.e., when you try to schedule
an action (event) at a time that has already passed? And, does
E << N lead to a different interpretation/handling?

There is no general answer to this: the only correct thing to do is
let the user specify how the event is to be treated if its scheduled
time is passed.

It reasonably is safe to assume that a "do it now" event should be
executed as soon as possible, even if was delayed several seconds in
the scheduling.

But beyond that you're speculating.


Unix 'cron', 'at', etc. are not particularly good examples to follow -
they are too simplistic. The set of options available to the Windows
scheduler is better (though not exhaustive), but IMO most of the
"options" should be mandatory parameters that must be provided in
order to schedule an event.

Post by Don Y
I've argued that the OS shouldn't special-case such activities.
If you request something to happen in the past, then the OS
should just act as if it has *just* happened, regardless as to
whether you were a microsecond "late" in issuing your request
or *years*! In particular, the OS shouldn't dismiss such request
unilaterally -- or, throw an error to alert the issuer to the
*apparent* inconsistency.

I think it should be an error for a /timed/ (not "now") event to be
scheduled past any possible execution time. An event that repeats
could be scheduled past its initial run time, but there should be at
least one repetition in the /future/.

E.g., an event that repeats 5 times at 1 day intervals starting 7 days
ago is meaningless now. But an event that repeats 5 times at 1 day
intervals starting /3/ days ago still has potential to execute.

Arguing that an administrator might (re)set the system time and that
might make a passed event relevant again is just wishful thinking.

[Don Y]

Post by George Neuner

Post by Don Y
N = "now"
R = task release time
D = task's deadline time
E = event time
The only real invariant is that R < N as code executing in a
task can't execute until the task has been released.
Specifically, there are no guaranteed relationship between
*most* of these times; N > E, N > D, D < R, etc. are all
possible in a generic system.
Though, in a nominal setting, R < N < E < D when the code tries
to initiate an event at a particular (future) time (E).
But, what happens when E < N -- i.e., when you try to schedule
an action (event) at a time that has already passed? And, does
E << N lead to a different interpretation/handling?

There is no general answer to this: the only correct thing to do is
let the user specify how the event is to be treated if its scheduled
time is passed.

I argue that, if the developer expects such a condition to
occur *or* there is a significant consequence to allowing
it to be unconditionally scheduled when he would prefer it
NOT be, then he should condition his invocation of the
event instead of burdening the syscall with yet another
option:

if !(now > event_time)
schedule(event_time)

This also draws attention to the fact that the event should
NOT be scheduled in that particular case -- in a more obvious
way than some parameter to an embelished syscall.

Post by George Neuner
It reasonably is safe to assume that a "do it now" event should be
executed as soon as possible, even if was delayed several seconds in
the scheduling.
But beyond that you're speculating.
Unix 'cron', 'at', etc. are not particularly good examples to follow -
they are too simplistic. The set of options available to the Windows
scheduler is better (though not exhaustive), but IMO most of the
"options" should be mandatory parameters that must be provided in
order to schedule an event.

Those are exposed to users. I'm looking at OS hooks that a developer
would exploit in an API (as above).

This opens the possibility of another class of potential errors:
schedule(event_time)
...
schedule(event_time+delta)
...
what if now > event_time + delta? If the developer had naively assumed
the first event would have completed within "delta" (instead of providing
a definite interlock on the scheduling of the second event), then you
could end up allowing both events to be "immediately" scheduled with
no clear indication of whether the first would complete before the
second got started. (i.e., a race baked into the implementation)

Post by George Neuner

Post by Don Y
I've argued that the OS shouldn't special-case such activities.
If you request something to happen in the past, then the OS
should just act as if it has *just* happened, regardless as to
whether you were a microsecond "late" in issuing your request
or *years*! In particular, the OS shouldn't dismiss such request
unilaterally -- or, throw an error to alert the issuer to the
*apparent* inconsistency.

I think it should be an error for a /timed/ (not "now") event to be
scheduled past any possible execution time. An event that repeats
could be scheduled past its initial run time, but there should be at
least one repetition in the /future/.

Run the speech recognizer's retraining algorithm at 01:00AM (because
no one is likely to be speaking, then). Ah, but shit happened and
we couldn't get around to it until 1:30... should we abort that?

If you treat the tasks in a system as being flexible in their
scheduling (which is inherent in almost all multitasking systems...
you can't be assured when ANY task *actually* executes), then you
can't define hard limits as to how "late" something can happen.

This is why I mentioned the "D"eadline in the times in my description.
I.e., if the task's deadline has passed, then does THAT impact
whether or not you should honor its request to schedule an activity?
Ans: Yes. If it is a hard deadline (which the developer specified!),
then the developer should know that it is possible for the task to
be aborted at any time prior to its completion. if the scheduling
was important, it shoul dbe handled in the task's deadline handler
with some interaction with the mainline code (did this get scheduled
before the task abended?)

Post by George Neuner
E.g., an event that repeats 5 times at 1 day intervals starting 7 days
ago is meaningless now. But an event that repeats 5 times at 1 day
intervals starting /3/ days ago still has potential to execute.
Arguing that an administrator might (re)set the system time and that
might make a passed event relevant again is just wishful thinking.

[George Neuner]

Post by Don Y

Post by George Neuner

Post by Don Y
N = "now"
R = task release time
D = task's deadline time
E = event time
The only real invariant is that R < N as code executing in a
task can't execute until the task has been released.
Specifically, there are no guaranteed relationship between
*most* of these times; N > E, N > D, D < R, etc. are all
possible in a generic system.
Though, in a nominal setting, R < N < E < D when the code tries
to initiate an event at a particular (future) time (E).
But, what happens when E < N -- i.e., when you try to schedule
an action (event) at a time that has already passed? And, does
E << N lead to a different interpretation/handling?

There is no general answer to this: the only correct thing to do is
let the user specify how the event is to be treated if its scheduled
time is passed.

I argue that, if the developer expects such a condition to
occur *or* there is a significant consequence to allowing
it to be unconditionally scheduled when he would prefer it
NOT be, then he should condition his invocation of the
event instead of burdening the syscall with yet another
if !(now > event_time)
schedule(event_time)
This also draws attention to the fact that the event should
NOT be scheduled in that particular case -- in a more obvious
way than some parameter to an embelished syscall.

Post by George Neuner
It reasonably is safe to assume that a "do it now" event should be
executed as soon as possible, even if was delayed several seconds in
the scheduling.
But beyond that you're speculating.
Unix 'cron', 'at', etc. are not particularly good examples to follow -
they are too simplistic. The set of options available to the Windows
scheduler is better (though not exhaustive), but IMO most of the
"options" should be mandatory parameters that must be provided in
order to schedule an event.

Those are exposed to users. I'm looking at OS hooks that a developer
would exploit in an API (as above).

Not the point. My comment was about what options / parameters are
available to the schedule(r).

Post by Don Y
schedule(event_time)
...
schedule(event_time+delta)
...
what if now > event_time + delta? If the developer had naively assumed
the first event would have completed within "delta" (instead of providing
a definite interlock on the scheduling of the second event), then you
could end up allowing both events to be "immediately" scheduled with
no clear indication of whether the first would complete before the
second got started. (i.e., a race baked into the implementation)

Post by George Neuner

Post by Don Y
I've argued that the OS shouldn't special-case such activities.
If you request something to happen in the past, then the OS
should just act as if it has *just* happened, regardless as to
whether you were a microsecond "late" in issuing your request
or *years*! In particular, the OS shouldn't dismiss such request
unilaterally -- or, throw an error to alert the issuer to the
*apparent* inconsistency.

I think it should be an error for a /timed/ (not "now") event to be
scheduled past any possible execution time. An event that repeats
could be scheduled past its initial run time, but there should be at
least one repetition in the /future/.

Run the speech recognizer's retraining algorithm at 01:00AM (because
no one is likely to be speaking, then). Ah, but shit happened and
we couldn't get around to it until 1:30... should we abort that?

Now you're not paying attention: I suggested above to look at the
Windows scheduler. One of the options (paraphrased) is "run asap if
missed".

But things like that should be the user / programmer choice based on
the task to be performed - not a system policy.

Post by Don Y
If you treat the tasks in a system as being flexible in their
scheduling (which is inherent in almost all multitasking systems...
you can't be assured when ANY task *actually* executes), then you
can't define hard limits as to how "late" something can happen.

Again the Windows scheduler: (paraphrased) there are options to
"wait <time units> for idle state"
"wait until in idle state for <time units>"
"stop if idle state ceases"
"start again if idle state resumes"
"after trigger delay execution randomly for <time units>"
"remove from schedule after <epoch>"
"check for network connections available"
"start only on line power"
"stop if on battery"
"wake up to run this task"
"keep trying to start for so many <time units>"
"stop/abort after so many <time units>"
"stop the task if it runs too long"
"force abort the task if it won't stop"

and more.

There are also /schedule/ priorities[*], and the task itself can be
scripted to run at a given OS priority (and as any particular user).
Sub "resource" for "idle" and this list ought to give you a few ideas
for what you should provide.


[*] schedule priority is not visible in the GUI. To see/modify it you
need to export the task to XML, edit the file and import it to
recreate the task with new settings. Yeah, Windows really /is/ a pain
sometimes.

Post by Don Y
This is why I mentioned the "D"eadline in the times in my description.
I.e., if the task's deadline has passed, then does THAT impact
whether or not you should honor its request to schedule an activity?
Ans: Yes. If it is a hard deadline (which the developer specified!),
then the developer should know that it is possible for the task to
be aborted at any time prior to its completion. if the scheduling
was important, it shoul dbe handled in the task's deadline handler
with some interaction with the mainline code (did this get scheduled
before the task abended?)

Post by George Neuner
E.g., an event that repeats 5 times at 1 day intervals starting 7 days
ago is meaningless now. But an event that repeats 5 times at 1 day
intervals starting /3/ days ago still has potential to execute.
Arguing that an administrator might (re)set the system time and that
might make a passed event relevant again is just wishful thinking.

[Don Y]

Post by George Neuner

Post by Don Y

Post by George Neuner
It reasonably is safe to assume that a "do it now" event should be
executed as soon as possible, even if was delayed several seconds in
the scheduling.
But beyond that you're speculating.
Unix 'cron', 'at', etc. are not particularly good examples to follow -
they are too simplistic. The set of options available to the Windows
scheduler is better (though not exhaustive), but IMO most of the
"options" should be mandatory parameters that must be provided in
order to schedule an event.

Those are exposed to users. I'm looking at OS hooks that a developer
would exploit in an API (as above).

Not the point. My comment was about what options / parameters are
available to the schedule(r).

Post by Don Y
schedule(event_time)
...
schedule(event_time+delta)
...
what if now > event_time + delta? If the developer had naively assumed
the first event would have completed within "delta" (instead of providing
a definite interlock on the scheduling of the second event), then you
could end up allowing both events to be "immediately" scheduled with
no clear indication of whether the first would complete before the
second got started. (i.e., a race baked into the implementation)

Post by George Neuner

Post by Don Y
I've argued that the OS shouldn't special-case such activities.
If you request something to happen in the past, then the OS
should just act as if it has *just* happened, regardless as to
whether you were a microsecond "late" in issuing your request
or *years*! In particular, the OS shouldn't dismiss such request
unilaterally -- or, throw an error to alert the issuer to the
*apparent* inconsistency.

I think it should be an error for a /timed/ (not "now") event to be
scheduled past any possible execution time. An event that repeats
could be scheduled past its initial run time, but there should be at
least one repetition in the /future/.

Run the speech recognizer's retraining algorithm at 01:00AM (because
no one is likely to be speaking, then). Ah, but shit happened and
we couldn't get around to it until 1:30... should we abort that?

Now you're not paying attention: I suggested above to look at the
Windows scheduler. One of the options (paraphrased) is "run asap if
missed".
But things like that should be the user / programmer choice based on
the task to be performed - not a system policy.

This goes beyond "system policy" (which would be some default
way to handle these types of incidents)

Post by George Neuner

Post by Don Y
I argue that, if the developer expects such a condition to
occur *or* there is a significant consequence to allowing
it to be unconditionally scheduled when he would prefer it
NOT be, then he should condition his invocation of the
event instead of burdening the syscall with yet another
if !(now > event_time)
schedule(event_time)
This also draws attention to the fact that the event should
NOT be scheduled in that particular case -- in a more obvious
way than some parameter to an embelished syscall.

The variety of different conditions that could be practical

Post by George Neuner

Post by Don Y
schedule(event_time)
...
schedule(event_time+delta)
...

one might condition the *second* schedule() with:

if (first_scheduled)
schedule(event_time+delta)

Post by George Neuner

Post by Don Y
If you treat the tasks in a system as being flexible in their
scheduling (which is inherent in almost all multitasking systems...
you can't be assured when ANY task *actually* executes), then you
can't define hard limits as to how "late" something can happen.

Again the Windows scheduler: (paraphrased) there are options to
"wait <time units> for idle state"
"wait until in idle state for <time units>"
"stop if idle state ceases"
"start again if idle state resumes"
"after trigger delay execution randomly for <time units>"
"remove from schedule after <epoch>"
"check for network connections available"
"start only on line power"
"stop if on battery"
"wake up to run this task"
"keep trying to start for so many <time units>"
"stop/abort after so many <time units>"
"stop the task if it runs too long"
"force abort the task if it won't stop"
and more.

And, no matter how many -- and how BLOATED the syscall becomes -- there
will STILL be conditions that are applicable to specific tasks.

Rather than trying to anticipate ALL of them to handle *in* the
syscall (and STILL having to account for others that were not
envisioned), it seems more prudent to have the syscall UNCONDITIONALLY
perform its action and let the developer assume responsibility for
writing explicit code to handle cases that *it* considers as "special".
Especially as bugs *there* are readily contained in the task's
environment and don't risk corrupting the kernel.

Post by George Neuner
There are also /schedule/ priorities[*], and the task itself can be
scripted to run at a given OS priority (and as any particular user).
Sub "resource" for "idle" and this list ought to give you a few ideas
for what you should provide.
[*] schedule priority is not visible in the GUI. To see/modify it you
need to export the task to XML, edit the file and import it to
recreate the task with new settings. Yeah, Windows really /is/ a pain
sometimes.

[George Neuner]

Post by Don Y

Post by George Neuner

Post by Don Y

Post by George Neuner
It reasonably is safe to assume that a "do it now" event should be
executed as soon as possible, even if was delayed several seconds in
the scheduling.
But beyond that you're speculating.
Unix 'cron', 'at', etc. are not particularly good examples to follow -
they are too simplistic. The set of options available to the Windows
scheduler is better (though not exhaustive), but IMO most of the
"options" should be mandatory parameters that must be provided in
order to schedule an event.

Those are exposed to users. I'm looking at OS hooks that a developer
would exploit in an API (as above).

Not the point. My comment was about what options / parameters are
available to the schedule(r).

Post by Don Y
schedule(event_time)
...
schedule(event_time+delta)
...
what if now > event_time + delta? If the developer had naively assumed
the first event would have completed within "delta" (instead of providing
a definite interlock on the scheduling of the second event), then you
could end up allowing both events to be "immediately" scheduled with
no clear indication of whether the first would complete before the
second got started. (i.e., a race baked into the implementation)

Post by George Neuner

Post by Don Y
I've argued that the OS shouldn't special-case such activities.
If you request something to happen in the past, then the OS
should just act as if it has *just* happened, regardless as to
whether you were a microsecond "late" in issuing your request
or *years*! In particular, the OS shouldn't dismiss such request
unilaterally -- or, throw an error to alert the issuer to the
*apparent* inconsistency.

I think it should be an error for a /timed/ (not "now") event to be
scheduled past any possible execution time. An event that repeats
could be scheduled past its initial run time, but there should be at
least one repetition in the /future/.

Run the speech recognizer's retraining algorithm at 01:00AM (because
no one is likely to be speaking, then). Ah, but shit happened and
we couldn't get around to it until 1:30... should we abort that?

Now you're not paying attention: I suggested above to look at the
Windows scheduler. One of the options (paraphrased) is "run asap if
missed".
But things like that should be the user / programmer choice based on
the task to be performed - not a system policy.

This goes beyond "system policy" (which would be some default
way to handle these types of incidents)

No I did /not/ miss this.

You asked how best to handle these things. The answer is provide
options in the scheduler and ensure that the options are (at least)
considered by the programmer by making them /required/ parameters.

If you don't like the number of parameters, pass them as a structure.

Post by Don Y

Post by George Neuner

Post by Don Y
I argue that, if the developer expects such a condition to
occur *or* there is a significant consequence to allowing
it to be unconditionally scheduled when he would prefer it
NOT be, then he should condition his invocation of the
event instead of burdening the syscall with yet another
if !(now > event_time)
schedule(event_time)
This also draws attention to the fact that the event should
NOT be scheduled in that particular case -- in a more obvious
way than some parameter to an embelished syscall.

The variety of different conditions that could be practical

Post by George Neuner

Post by Don Y
schedule(event_time)
...
schedule(event_time+delta)
...

if (first_scheduled)
schedule(event_time+delta)

Post by George Neuner

Post by Don Y
If you treat the tasks in a system as being flexible in their
scheduling (which is inherent in almost all multitasking systems...
you can't be assured when ANY task *actually* executes), then you
can't define hard limits as to how "late" something can happen.

What you're describing here is meta: it's a condition of utilizing the
scheduler in the 1st place, not a condition of triggering a scheduled
event.

I think what you may really /mean/ is dependency scheduling ... i.e.
"this event depends on there being one or more other events already
scheduled".

The number and criticality of such dependencies can be unbounded - or
at least unworkable to specify - and I don't think checking for them
belongs /in/ the scheduler or its API. If some program depends on
such meta knowledge of what is scheduled, it should have to keep track
of that itself.

Post by Don Y

Post by George Neuner
Again the Windows scheduler: (paraphrased) there are options to
"wait <time units> for idle state"
"wait until in idle state for <time units>"
"stop if idle state ceases"
"start again if idle state resumes"
"after trigger delay execution randomly for <time units>"
"remove from schedule after <epoch>"
"check for network connections available"
"start only on line power"
"stop if on battery"
"wake up to run this task"
"keep trying to start for so many <time units>"
"stop/abort after so many <time units>"
"stop the task if it runs too long"
"force abort the task if it won't stop"
and more.

And, no matter how many -- and how BLOATED the syscall becomes -- there
will STILL be conditions that are applicable to specific tasks.
Rather than trying to anticipate ALL of them to handle *in* the
syscall (and STILL having to account for others that were not
envisioned), it seems more prudent to have the syscall UNCONDITIONALLY
perform its action and let the developer assume responsibility for
writing explicit code to handle cases that *it* considers as "special".
Especially as bugs *there* are readily contained in the task's
environment and don't risk corrupting the kernel.

If all the options can be (de)selected individually, then you don't
have to bundle them all together ... but then you run the risk of the
programmer forgetting something.

It's a good idea anyway for the API to allow modifying an existing
schedule without having to delete/recreate it (even if "modify" really
is just an illusion for the programmer).


Scheduling in a complex system is NOT SIMPLE and it never has been:
whole books have been written about it. The overwhelming majority of
events likely will require only a small subset of the scheduler's
options, but the scheduler itself must deal with the most complex case
possible.

You might consider having multiple APIs [parameter blocks?] of varying
complexity. But there must be some version of the API that can
specify every possible option.

Post by Don Y

Post by George Neuner
There are also /schedule/ priorities[*], and the task itself can be
scripted to run at a given OS priority (and as any particular user).
Sub "resource" for "idle" and this list ought to give you a few ideas
for what you should provide.
[*] schedule priority is not visible in the GUI. To see/modify it you
need to export the task to XML, edit the file and import it to
recreate the task with new settings. Yeah, Windows really /is/ a pain
sometimes.

[Don Y]

Post by George Neuner
You asked how best to handle these things. The answer is provide
options in the scheduler and ensure that the options are (at least)
considered by the programmer by making them /required/ parameters.
If you don't like the number of parameters, pass them as a structure.

I object to the *complexity* that all those different potential
conditionals impose on the facility.

If you want a fancy/smart scheduler interface, build a *service*
that implements it and let *that* talk to the actual scheduler.
If the service fails, then any clients that RELY on it get screwed,
but the REAL scheduler still keeps working (for ITS clients)

Post by George Neuner

Post by Don Y

Post by George Neuner

Post by Don Y
If you treat the tasks in a system as being flexible in their
scheduling (which is inherent in almost all multitasking systems...
you can't be assured when ANY task *actually* executes), then you
can't define hard limits as to how "late" something can happen.

What you're describing here is meta: it's a condition of utilizing the
scheduler in the 1st place, not a condition of triggering a scheduled
event.

Exactly. Why tell the scheduler how to respond to changes in the
environment -- changes that you can never completely anticipate
in designing an API.

Let the application figure out what "makes sense" for it.

If the time for the event/activity is passed and that is inconsequential
to the scheduling of that activity, then go ahead and unconditionally
schedule it.

If, however, there is some (possibly complex) relationship between
the CURRENT environment and the ANTICIPATED environment, let the
developer sort out the significance of these differences and
make a conscious decision as to how to proceed, if at all.

Post by George Neuner
I think what you may really /mean/ is dependency scheduling ... i.e.
"this event depends on there being one or more other events already
scheduled".

No, it is a reflection of how "now" has changed from what the
developer THOUGHT "now" would be like when that line of code
(schedule(x)) was about to be executed.

E.g., the event_time+delta clearly reflects an assumption that the
previous event -- at event_time -- will have completed before the
next event occurs. Unless there is some other mechanism in place
to assure this, it is likely a faulty assumption.

So, write the code ignorant of that possibility and address the
bug WHEN it manifests.

OR, redesign the implementation so that implicit expectation
is made more explicit. E.g., possibly require the first event
to set a flag to indicate that it HAS completed before you
allow the second event to proceed (even if both have been skewed
significantly in time).

Post by George Neuner
The number and criticality of such dependencies can be unbounded - or
at least unworkable to specify - and I don't think checking for them
belongs /in/ the scheduler or its API. If some program depends on
such meta knowledge of what is scheduled, it should have to keep track
of that itself.

That is my point. schedule(X) simply and unconditionally schedules X.
If the task abends BEFORE that line of code executes, then it is the
developer's job to figure out how <whatever> has been impacted by
that non-event. E.g., if it NEEDS to be scheduled, regardless, then
redesign the application to make that guarantee explicit and obvious.

If the scheduled time PASSES before that line of code executes,
then, similarly, it is up to the developer to decide how it
should be handled. E.g., if the activity only makes sense at
that particular point in time, then you don't want to schedule
it if that point in time is no longer attainable.

So, schedule() is freed from being concerned with these issues.

If you want an "agent" to implement some particular policy on your behalf,
then create one; spawn another task that enforces that policy in
explicit code (so a maintainer sees that this is an important
aspect of the algorithm)

Post by George Neuner

Post by Don Y

Post by George Neuner
Again the Windows scheduler: (paraphrased) there are options to
"wait <time units> for idle state"
"wait until in idle state for <time units>"
"stop if idle state ceases"
"start again if idle state resumes"
"after trigger delay execution randomly for <time units>"
"remove from schedule after <epoch>"
"check for network connections available"
"start only on line power"
"stop if on battery"
"wake up to run this task"
"keep trying to start for so many <time units>"
"stop/abort after so many <time units>"
"stop the task if it runs too long"
"force abort the task if it won't stop"
and more.

And, no matter how many -- and how BLOATED the syscall becomes -- there
will STILL be conditions that are applicable to specific tasks.
Rather than trying to anticipate ALL of them to handle *in* the
syscall (and STILL having to account for others that were not
envisioned), it seems more prudent to have the syscall UNCONDITIONALLY
perform its action and let the developer assume responsibility for
writing explicit code to handle cases that *it* considers as "special".
Especially as bugs *there* are readily contained in the task's
environment and don't risk corrupting the kernel.

If all the options can be (de)selected individually, then you don't
have to bundle them all together ... but then you run the risk of the
programmer forgetting something.
It's a good idea anyway for the API to allow modifying an existing
schedule without having to delete/recreate it (even if "modify" really
is just an illusion for the programmer).

It's just another object. Anyone (including some OTHER task)
with an appropriate capability (set) can invoke whatever options
they deem necessary. E.g., the deadline handler can use that
handle to delete the scheduled event if *it* determines that
it doesn't make sense to allow it to continue.

Of course, that only is possible if the event hasn't already been
triggered.

Post by George Neuner
whole books have been written about it. The overwhelming majority of
events likely will require only a small subset of the scheduler's
options, but the scheduler itself must deal with the most complex case
possible.

This is why it is easier (to get "right") a simple, unconditional
syscall that doesn't have to evaluate other conditions -- ASSUME the
caller had already done that work and DECIDED that the event should
be scheduled. So, you ("I") can concentrate on JUST getting the event
scheduled.

Post by George Neuner
You might consider having multiple APIs [parameter blocks?] of varying
complexity. But there must be some version of the API that can
specify every possible option.

Post by Don Y

Post by George Neuner
There are also /schedule/ priorities[*], and the task itself can be
scripted to run at a given OS priority (and as any particular user).
Sub "resource" for "idle" and this list ought to give you a few ideas
for what you should provide.
[*] schedule priority is not visible in the GUI. To see/modify it you
need to export the task to XML, edit the file and import it to
recreate the task with new settings. Yeah, Windows really /is/ a pain
sometimes.

[George Neuner]

Post by Don Y

Post by George Neuner
You asked how best to handle these things. The answer is provide
options in the scheduler and ensure that the options are (at least)
considered by the programmer by making them /required/ parameters.
If you don't like the number of parameters, pass them as a structure.

I object to the *complexity* that all those different potential
conditionals impose on the facility.
If you want a fancy/smart scheduler interface, build a *service*
that implements it and let *that* talk to the actual scheduler.
If the service fails, then any clients that RELY on it get screwed,
but the REAL scheduler still keeps working (for ITS clients)

The problem is that the REAL scheduler /needs/ the flexibility of all
of those options to do its job. You can put lipstick on the pig - ie.
use a service to present a more friendly use API - but that doesn't
change the underlying requirements.

Post by Don Y

Post by George Neuner

Post by Don Y
If you treat the tasks in a system as being flexible in their
scheduling (which is inherent in almost all multitasking systems...
you can't be assured when ANY task *actually* executes), then you
can't define hard limits as to how "late" something can happen.

What you're describing here is meta: it's a condition of utilizing the
scheduler in the 1st place, not a condition of triggering a scheduled
event.

Exactly. Why tell the scheduler how to respond to changes in the
environment -- changes that you can never completely anticipate
in designing an API.
Let the application figure out what "makes sense" for it.

The problem with that is that it ends up putting a whole bunch of what
/effectively/ is - or should be - "scheduler code" into a whole bunch
of applications. Instead of reducing application complexity by
introducing a smart service, you instead increase complexity by
forcing the application to figure out for itself when and how it can
make use of a dumb service.

Post by Don Y

Post by George Neuner
I think what you may really /mean/ is dependency scheduling ... i.e.
"this event depends on there being one or more other events already
scheduled".

No, it is a reflection of how "now" has changed from what the
developer THOUGHT "now" would be like when that line of code
(schedule(x)) was about to be executed.
E.g., the event_time+delta clearly reflects an assumption that the
previous event -- at event_time -- will have completed before the
next event occurs. Unless there is some other mechanism in place
to assure this, it is likely a faulty assumption.

Where do you get that from ?

Post by Don Y

Post by George Neuner
schedule(event_time)
...
schedule(event_time+delta)

All you can reasonably conclude from this fragment is that the
scheduler was invoked twice: first on the 1st event, then on the 2nd -
and that assuming they were scheduled successfully, the 2nd event can
be expected to be triggered at least "delta" time units after the 1st
is triggered.

Nothing in this indicates that "delta" is intended to allow the 1st
event to complete whatever it was supposed to do. In fact, nothing
indicates the two events even are related.


Which is precisely the point I was making about scheduling options.
You have to have at least some way to deal with the relationship. If
the 2nd depends on the 1st, then the tasks must signal outside the
scheduler, and the 2nd potentially must fail with a code the scheduler
understands means to "try again". And then there must be options to
say how long before trying again, how many times to try again before
giving up, etc.

Post by Don Y
So, write the code ignorant of that possibility and address the
bug WHEN it manifests.

You're assuming a particular style of coding ... I don't think you can
make that assumption.

Post by Don Y
OR, redesign the implementation so that implicit expectation
is made more explicit. E.g., possibly require the first event
to set a flag to indicate that it HAS completed before you
allow the second event to proceed (even if both have been skewed
significantly in time).

Again, that is outside the pervue of the scheduler. If A wants
somehow to signal B that it has completed, that has to be handled by
the tasks themselves.

Unless the scheduler can know that A has completed, and know the
status of A when it finished ... and make executing B dependent on
that. But that is asking too much because:

I wrote previously -

Post by Don Y

Post by George Neuner
The number and criticality of such dependencies can be unbounded - or
at least unworkable to specify - and I don't think checking for them
belongs /in/ the scheduler or its API. If some program depends on
such meta knowledge of what is scheduled, it should have to keep track
of that itself.

That is my point. schedule(X) simply and unconditionally schedules X.
If the task abends BEFORE that line of code executes, then it is the
developer's job to figure out how <whatever> has been impacted by
that non-event. E.g., if it NEEDS to be scheduled, regardless, then
redesign the application to make that guarantee explicit and obvious.
If the scheduled time PASSES before that line of code executes,
then, similarly, it is up to the developer to decide how it
should be handled. E.g., if the activity only makes sense at
that particular point in time, then you don't want to schedule
it if that point in time is no longer attainable.
So, schedule() is freed from being concerned with these issues.

If you've already made up your mind that that is the way it must work,
then what the heck is this conversation about?

Post by Don Y

Post by George Neuner
whole books have been written about it. The overwhelming majority of
events likely will require only a small subset of the scheduler's
options, but the scheduler itself must deal with the most complex case
possible.

This is why it is easier (to get "right") a simple, unconditional
syscall that doesn't have to evaluate other conditions -- ASSUME the
caller had already done that work and DECIDED that the event should
be scheduled. So, you ("I") can concentrate on JUST getting the event
scheduled.

[Don Y]

Post by George Neuner

Post by Don Y

Post by George Neuner
You asked how best to handle these things. The answer is provide
options in the scheduler and ensure that the options are (at least)
considered by the programmer by making them /required/ parameters.
If you don't like the number of parameters, pass them as a structure.

I object to the *complexity* that all those different potential
conditionals impose on the facility.
If you want a fancy/smart scheduler interface, build a *service*
that implements it and let *that* talk to the actual scheduler.
If the service fails, then any clients that RELY on it get screwed,
but the REAL scheduler still keeps working (for ITS clients)

The problem is that the REAL scheduler /needs/ the flexibility of all
of those options to do its job. You can put lipstick on the pig - ie.
use a service to present a more friendly use API - but that doesn't
change the underlying requirements.

No, it doesn't. The scheduler just has to make a task ready for
execution NO SOONER THAN a particular time.

E.g., if you write:
delay(1000)
all you know is an interval OF AT LEAST 1000 will occur before
the next statement executes. It may be longer -- if there are
competing needs -- but likely won't be EXACTLY 1000.

Guaranteeing a particular time is impossible -- because you don't
know *if* the request will be made prior to that time (or not)
and how much "other work" will compete for the processor.

If guaranteeing the time for one event is impossible, then how
is it possible to guarantee the time for a *subsequent* event
(envisioned when the code was *written*, not executed).

If you want a particular (set of) condition(s) to apply to the
task's execution, then STATE them. Delegating those things to some
other service/facility constrains what you can do with that
facility. So, in some places, you rely on it and in others
you augment it.

If it can't handle EVERY condition, why burden it with handling *any*?

I've gone through each of the "special cases" that have been presented
to me and shown how to get the behavior ACTUALLY sought by conditioning
such invocations. Keeping the scheduler simple (so it can concentrate
on getting all of the *resources* that a task will need "on line"
so the task *can* be dispatched) eliminates complexity that often
would not address real needs (but will allow opportunities for
faults to manifest in the code).

It also makes it easier to track the actual resources used by
a task (that is scheduling) because you can watch while *it*
is executing -- instead of having to figure out how much of the
scheduler's activities (at a possible later time) are "chargeable"
to that task.

[George Neuner]

Post by Don Y

Post by George Neuner

Post by Don Y

Post by George Neuner
You asked how best to handle these things. The answer is provide
options in the scheduler and ensure that the options are (at least)
considered by the programmer by making them /required/ parameters.
If you don't like the number of parameters, pass them as a structure.

I object to the *complexity* that all those different potential
conditionals impose on the facility.
If you want a fancy/smart scheduler interface, build a *service*
that implements it and let *that* talk to the actual scheduler.
If the service fails, then any clients that RELY on it get screwed,
but the REAL scheduler still keeps working (for ITS clients)

The problem is that the REAL scheduler /needs/ the flexibility of all
of those options to do its job. You can put lipstick on the pig - ie.
use a service to present a more friendly use API - but that doesn't
change the underlying requirements.

No, it doesn't. The scheduler just has to make a task ready for
execution NO SOONER THAN a particular time.
delay(1000)
all you know is an interval OF AT LEAST 1000 will occur before
the next statement executes. It may be longer -- if there are
competing needs -- but likely won't be EXACTLY 1000.
Guaranteeing a particular time is impossible -- because you don't
know *if* the request will be made prior to that time (or not)
and how much "other work" will compete for the processor.
If guaranteeing the time for one event is impossible, then how
is it possible to guarantee the time for a *subsequent* event
(envisioned when the code was *written*, not executed).
If you want a particular (set of) condition(s) to apply to the
task's execution, then STATE them. Delegating those things to some
other service/facility constrains what you can do with that
facility. So, in some places, you rely on it and in others
you augment it.
If it can't handle EVERY condition, why burden it with handling *any*?

Say you want a task to start every day at 00, 08 and 16. You don't
care that it starts exactly on time, but you don't want the ideal
start time to drift with execution, so (re)scheduling as part of the
task won't do.

Say you want a task that runs M,W,F at 10, and T,Th,Sa at 14. Sunday
it does not run. Again, you don't care that it runs exactly on time,
but it must complete within an hour and so it must start within 15
minutes of the scheduled time or it can't be run.


Are you going to run a scheduling task every day to schedule for
tomorrow? What if that task doesn't get run?

Are you going to run one or more monitor tasks constantly (wasting CPU
and power) to schedule other tasks?

IMO either of these are ridiculous.


You want a schedule that starts at 00, repeats every 8 hours, and
(pending resources) continues to try to start for X <time>.

You want a schedule that runs M,W,F at 10, tries to start for up to 15
minutes, then aborts if it can't be started. You want a second
instance of this schedule that runs T,Th,Sa at 14.

And you want options in your /scheduler/ to be able to specify these
conditions.

Post by Don Y
I've gone through each of the "special cases" that have been presented
to me and shown how to get the behavior ACTUALLY sought by conditioning
such invocations. Keeping the scheduler simple (so it can concentrate
on getting all of the *resources* that a task will need "on line"
so the task *can* be dispatched) eliminates complexity that often
would not address real needs (but will allow opportunities for
faults to manifest in the code).
It also makes it easier to track the actual resources used by
a task (that is scheduling) because you can watch while *it*
is executing -- instead of having to figure out how much of the
scheduler's activities (at a possible later time) are "chargeable"
to that task.

[Don Y]

Post by George Neuner

Post by Don Y

Post by George Neuner

Post by Don Y

Post by George Neuner
You asked how best to handle these things. The answer is provide
options in the scheduler and ensure that the options are (at least)
considered by the programmer by making them /required/ parameters.
If you don't like the number of parameters, pass them as a structure.

I object to the *complexity* that all those different potential
conditionals impose on the facility.
If you want a fancy/smart scheduler interface, build a *service*
that implements it and let *that* talk to the actual scheduler.
If the service fails, then any clients that RELY on it get screwed,
but the REAL scheduler still keeps working (for ITS clients)

The problem is that the REAL scheduler /needs/ the flexibility of all
of those options to do its job. You can put lipstick on the pig - ie.
use a service to present a more friendly use API - but that doesn't
change the underlying requirements.

No, it doesn't. The scheduler just has to make a task ready for
execution NO SOONER THAN a particular time.
delay(1000)
all you know is an interval OF AT LEAST 1000 will occur before
the next statement executes. It may be longer -- if there are
competing needs -- but likely won't be EXACTLY 1000.
Guaranteeing a particular time is impossible -- because you don't
know *if* the request will be made prior to that time (or not)
and how much "other work" will compete for the processor.
If guaranteeing the time for one event is impossible, then how
is it possible to guarantee the time for a *subsequent* event
(envisioned when the code was *written*, not executed).
If you want a particular (set of) condition(s) to apply to the
task's execution, then STATE them. Delegating those things to some
other service/facility constrains what you can do with that
facility. So, in some places, you rely on it and in others
you augment it.
If it can't handle EVERY condition, why burden it with handling *any*?

Say you want a task to start every day at 00, 08 and 16. You don't
care that it starts exactly on time, but you don't want the ideal
start time to drift with execution, so (re)scheduling as part of the
task won't do.
Say you want a task that runs M,W,F at 10, and T,Th,Sa at 14. Sunday
it does not run. Again, you don't care that it runs exactly on time,
but it must complete within an hour and so it must start within 15
minutes of the scheduled time or it can't be run.

You schedule *one* task that schedules the "real" task, based
on it's dynamic observations of the current state of the system.

If the Monday task couldn't run at 10 -- because of a power outage -- should
it be rescheduled to run when power resumes? Should it be ignored because
the Tuesday task will effectively address the issues that were NOT
addressed by the Monday (non)invocation?

Post by George Neuner
Are you going to run a scheduling task every day to schedule for
tomorrow? What if that task doesn't get run?

Exactly. See above.

What if the *scheduler* doesn't run? Do ALL tasks in the system that
it was responsible for initiating stop running?

Post by George Neuner
Are you going to run one or more monitor tasks constantly (wasting CPU
and power) to schedule other tasks?

A task that idles consumes few resources. Is the "scheduler"
wasteful of CPU and power?

Post by George Neuner
IMO either of these are ridiculous.
You want a schedule that starts at 00, repeats every 8 hours, and
(pending resources) continues to try to start for X <time>.
You want a schedule that runs M,W,F at 10, tries to start for up to 15
minutes, then aborts if it can't be started. You want a second
instance of this schedule that runs T,Th,Sa at 14.
And you want options in your /scheduler/ to be able to specify these
conditions.

No. That complicates the scheduler to handle *arbitrarily* complex cases
("schedule this if it has NOT rained", "schedule this if there have
been visitors", "schedule this if there has been a power outage",
"schedule this if the database needs to be vacuumed", etc.).

Let the "parent" of the task sort out when it should run and
write explicit code to test for those conditions. If that
code is buggy, then *it's* scheduling is buggy -- but not the
scheduler itself.

Post by George Neuner

Post by Don Y
I've gone through each of the "special cases" that have been presented
to me and shown how to get the behavior ACTUALLY sought by conditioning
such invocations. Keeping the scheduler simple (so it can concentrate
on getting all of the *resources* that a task will need "on line"
so the task *can* be dispatched) eliminates complexity that often
would not address real needs (but will allow opportunities for
faults to manifest in the code).
It also makes it easier to track the actual resources used by
a task (that is scheduling) because you can watch while *it*
is executing -- instead of having to figure out how much of the
scheduler's activities (at a possible later time) are "chargeable"
to that task.

Do note that the question *posed* was:
how to handle a scheduled event time when said time was
in the recent -- or distant -- past.

[Waldek Hebisch]

Post by Don Y
N = "now"
R = task release time
D = task's deadline time
E = event time
The only real invariant is that R < N as code executing in a
task can't execute until the task has been released.
Specifically, there are no guaranteed relationship between
*most* of these times; N > E, N > D, D < R, etc. are all
possible in a generic system.
Though, in a nominal setting, R < N < E < D when the code tries
to initiate an event at a particular (future) time (E).
But, what happens when E < N -- i.e., when you try to schedule
an action (event) at a time that has already passed? And, does
E << N lead to a different interpretation/handling?
I've argued that the OS shouldn't special-case such activities.
If you request something to happen in the past, then the OS
should just act as if it has *just* happened, regardless as to
whether you were a microsecond "late" in issuing your request
or *years*! In particular, the OS shouldn't dismiss such request
unilaterally -- or, throw an error to alert the issuer to the
*apparent* inconsistency.

That is confusing formulation of the problem. First, what is
"the OS"? Second, your "task release time", "task's deadline time",
"event time" are undefined. In particular, one meaning of
deadline is "time when a task should stop, regarless if it is
done or not".

If by "the OS" you mean operating system kernel, then you want
it as simple as possible and move various functions to upper
layers. In particular, in general purpose OS you may have
timers which activate a task some time after scheduled time
(hopefully close to scheduled time, but there is no warranty).
Maybe you are thinking of such a timer.

However, there may be higher level "scheduler". For example
Unix 'cron' and 'at' are userspace programs, but are considered
part of operating system. 'cron' and 'at' are rather crude,
if you may need more. In particular, an automations system
typically must schedule various activities with specific time
constraints.

Anyway, your system probably is intended to do something useful
and that will lead to some requirements. Once you have more
concerte requirements, you can think how to best satisfy them.
If you have only handful of time-dependent tasks and each
differs a lot from other, then putting all higher level
time handling in the tasks may be reasonable. But usually
it pays to have common handling. As George mentioned there
can be interdependencies between task, so it is likely that
you may need a higher level scheduler to coordinate them.
Concerning kernel, you should have in kernel enough to
support higher level code. In particular, if you have
"fast" tasks and care about their time performance, then
you may need real-time features in the kernel. If you do
not care, than usual general-purpose approach is enough.
But if you do not care, then why are you talking about
deadlines?

Concerning tasks way in the past: this is very unusual case
for general purpose kernel. Namely, the only likely reason
for long delay is because system was turned off. But kernel
timers frequently are "volatile", that is they will be gone
when system is turned on again (if that is undesirable, then
higher level layer may restore timers that were supposed to
persist). AFAIK general purpose kernels handled timers
as you argue, that is dully activated corresponding task.
OTOH long delays were usually delegated to userspace which
could have complicated rules.

[Don Y]

Post by Waldek Hebisch

Post by Don Y
N = "now"
R = task release time
D = task's deadline time
E = event time
The only real invariant is that R < N as code executing in a
task can't execute until the task has been released.
Specifically, there are no guaranteed relationship between
*most* of these times; N > E, N > D, D < R, etc. are all
possible in a generic system.
Though, in a nominal setting, R < N < E < D when the code tries
to initiate an event at a particular (future) time (E).
But, what happens when E < N -- i.e., when you try to schedule
an action (event) at a time that has already passed? And, does
E << N lead to a different interpretation/handling?
I've argued that the OS shouldn't special-case such activities.
If you request something to happen in the past, then the OS
should just act as if it has *just* happened, regardless as to
whether you were a microsecond "late" in issuing your request
or *years*! In particular, the OS shouldn't dismiss such request
unilaterally -- or, throw an error to alert the issuer to the
*apparent* inconsistency.

That is confusing formulation of the problem. First, what is
"the OS"?

Doesn't matter. The question is about how one should EXPECT an OS
to handle such things.

Post by Waldek Hebisch
Second, your "task release time", "task's deadline time",
"event time" are undefined. In particular, one meaning of
deadline is "time when a task should stop, regarless if it is
done or not".

"Release time" is the time a task is made eligible to run.

"Deadline time" is the time at which the value of a task's
completion, changes. I.e., this is the time that you
strive to meet -- but the consequences of failing to meet
it are dependent on the nature of the task and how "important"
you consider its completion BY that time to be.

In particular, you can make no assumptions about whether or
not the task should stop once that time has passed. For this
discussion, you can assume that all deadlines are "hard"
(to simplify the argument; if NOW is past the deadline, then
the task is moribund)

"Event time" is the time that you would like "something" to happen.

Post by Waldek Hebisch
If by "the OS" you mean operating system kernel, then you want
it as simple as possible and move various functions to upper
layers. In particular, in general purpose OS you may have
timers which activate a task some time after scheduled time
(hopefully close to scheduled time, but there is no warranty).
Maybe you are thinking of such a timer.

I am thinking one layer above that, in terms of abstraction.
Imagine some concurrent agency "watching the clock" and
taking action on your request to make something happen at a
particular time. *YOU* may be busy with something else.

Post by Waldek Hebisch
However, there may be higher level "scheduler". For example
Unix 'cron' and 'at' are userspace programs, but are considered
part of operating system. 'cron' and 'at' are rather crude,
if you may need more. In particular, an automations system
typically must schedule various activities with specific time
constraints.
Anyway, your system probably is intended to do something useful
and that will lead to some requirements. Once you have more
concerte requirements, you can think how to best satisfy them.
If you have only handful of time-dependent tasks and each
differs a lot from other, then putting all higher level
time handling in the tasks may be reasonable. But usually
it pays to have common handling. As George mentioned there
can be interdependencies between task, so it is likely that
you may need a higher level scheduler to coordinate them.

My contention is that said service can't possibly know of all
of the interdependencies that could exist (examples provided
in other posts). So, let the service focus on a single criteria:
the time at which you desire the action to occur.

Consider, however, that the service may be displaced by some higher
priority tasks executing on its hardware. So, it can't guarantee
that it ever *sees* the specific time you've specified; other
tasks may have preempted it. So, expecting to hit a specific
time is wishful thinking.

Also, a specific time may not be appropriate for code written
some days/months/years earlier. For example, maybe I want to
ensure a task is activated when Daylight Savings Time ("Summer
Time") takes effect. So, that time could be hard-coded into
a task without regard for when the task was initially invoked.

[There are lots of fixed dates/times in our lives. So, these
could be unconditionally encoded without regard for *when*
that encoding took place]

Post by Waldek Hebisch
Concerning kernel, you should have in kernel enough to
support higher level code. In particular, if you have
"fast" tasks and care about their time performance, then
you may need real-time features in the kernel. If you do
not care, than usual general-purpose approach is enough.
But if you do not care, then why are you talking about
deadlines?

Because deadlines are always important. Even non-realtime
activities have some sense of timeliness.

I don't really care *when* the results of my checkbook balancing
program are available for inspection. But, I /probably/ would
like them before I write my next check! :>

Deadlines and value/importance let you (as a human) and
the machine decide where your efforts should be focused.

E.g., if you miss your spouse's birthday, should you still
attempt to recognize/celebrate it, even if late? What if
you miss your NEIGHBOR's birthday??

Post by Waldek Hebisch
Concerning tasks way in the past: this is very unusual case
for general purpose kernel. Namely, the only likely reason
for long delay is because system was turned off.

Note that I deliberately tried to address "distant past"
vs. "not so distant past". But, have left the magnitudes of
these differences unspecified.

"Not so distant past" seems like it is a condition that could
readily happen. E.g., if you tried to schedule an event but
found yourself delayed (by competing tasks) so that the scheduled
time was upon you before you got a chance to execute that function.
"Soooooo close!"

Given that this can happen, how "close" to missing that time
should you consider "close enough" that you can pretend you
managed to schedule it before the specified time. In which
case, one could argue that it should be activated "now"
(because "now" is really close to "then")

Post by Waldek Hebisch
But kernel
timers frequently are "volatile", that is they will be gone
when system is turned on again (if that is undesirable, then
higher level layer may restore timers that were supposed to
persist). AFAIK general purpose kernels handled timers
as you argue, that is dully activated corresponding task.
OTOH long delays were usually delegated to userspace which
could have complicated rules.

You have to assume these are unprivileged (out-of-kernel)
tasks -- because you don't want to bring more code *into*
the kernel.

But, their importance is something you can't know, a priori
(you didn't write them!)

And, you can't determine if your chosen method of handling
"past" times will break the task that is making the request,
or not.

E.g., I can schedule a task to retrain my speech recognizer
in the wee hours of the morning (when the system load is lower),
based on input it has collected over the course of the day.
If I miss this time, it means that the quality of the recognizer
"tomorrow" will be less than it could have been. But, if the
retraining can happen TOMORROW night (again, in the system's
"off" time), then there isn't any significant loss.

OTOH, if I allow the retraining to happen at some delayed time that
interferes with other system tasks ("prime time"), other activities
could be compromised.

So, I argue that a simple criteria be adopted and the writer of
the task requesting the scheduling (and the task being scheduled)
address their own specific criteria IN EXPLICIT CODE to enforce
these, instead of hoping some other service can be made aware of
these requirements (by embelishing that service? by including
support fr up-calls to the task doing the scheduling??)

The "scheduler" can then just focus on ensuring each event gets
processed exactly once and at a time NO EARLIER than the time
specified. This is much easier for a developer to wrap his head
around than trying to second-guess the scheduler ("What if the
scheduler is blocked for some period of time? How will *it*
handle things when NOW is later than it *could* have been -- or,
when there are MANY competing things in the queue?")

[Waldek Hebisch]

Post by Don Y

Post by Waldek Hebisch

Post by Don Y
N = "now"
R = task release time
D = task's deadline time
E = event time
The only real invariant is that R < N as code executing in a
task can't execute until the task has been released.
Specifically, there are no guaranteed relationship between
*most* of these times; N > E, N > D, D < R, etc. are all
possible in a generic system.
Though, in a nominal setting, R < N < E < D when the code tries
to initiate an event at a particular (future) time (E).
But, what happens when E < N -- i.e., when you try to schedule
an action (event) at a time that has already passed? And, does
E << N lead to a different interpretation/handling?
I've argued that the OS shouldn't special-case such activities.
If you request something to happen in the past, then the OS
should just act as if it has *just* happened, regardless as to
whether you were a microsecond "late" in issuing your request
or *years*! In particular, the OS shouldn't dismiss such request
unilaterally -- or, throw an error to alert the issuer to the
*apparent* inconsistency.

That is confusing formulation of the problem. First, what is
"the OS"?

Doesn't matter. The question is about how one should EXPECT an OS
to handle such things.

It matters. OS kernel should implement something simple, but enough
to support higher layers. At higher layers you may have something
more fancy.

Post by Don Y

Post by Waldek Hebisch
Second, your "task release time", "task's deadline time",
"event time" are undefined. In particular, one meaning of
deadline is "time when a task should stop, regarless if it is
done or not".

"Release time" is the time a task is made eligible to run.
"Deadline time" is the time at which the value of a task's
completion, changes. I.e., this is the time that you
strive to meet -- but the consequences of failing to meet
it are dependent on the nature of the task and how "important"
you consider its completion BY that time to be.
In particular, you can make no assumptions about whether or
not the task should stop once that time has passed. For this
discussion, you can assume that all deadlines are "hard"
(to simplify the argument; if NOW is past the deadline, then
the task is moribund)
"Event time" is the time that you would like "something" to happen.

OK

Post by Don Y

Post by Waldek Hebisch
If by "the OS" you mean operating system kernel, then you want
it as simple as possible and move various functions to upper
layers. In particular, in general purpose OS you may have
timers which activate a task some time after scheduled time
(hopefully close to scheduled time, but there is no warranty).
Maybe you are thinking of such a timer.

I am thinking one layer above that, in terms of abstraction.
Imagine some concurrent agency "watching the clock" and
taking action on your request to make something happen at a
particular time. *YOU* may be busy with something else.

I take this and what you write later that you think about
library interface for "applications".

Post by Don Y

Post by Waldek Hebisch
However, there may be higher level "scheduler". For example
Unix 'cron' and 'at' are userspace programs, but are considered
part of operating system. 'cron' and 'at' are rather crude,
if you may need more. In particular, an automations system
typically must schedule various activities with specific time
constraints.
Anyway, your system probably is intended to do something useful
and that will lead to some requirements. Once you have more
concerte requirements, you can think how to best satisfy them.
If you have only handful of time-dependent tasks and each
differs a lot from other, then putting all higher level
time handling in the tasks may be reasonable. But usually
it pays to have common handling. As George mentioned there
can be interdependencies between task, so it is likely that
you may need a higher level scheduler to coordinate them.

My contention is that said service can't possibly know of all
of the interdependencies that could exist (examples provided
the time at which you desire the action to occur.

You can not solve _all_ scheduling problems. It does not
mean that you should not solve some. Windows style interface
where you have a lot of specific cases and can not do anything
else than provided choices does not look attractive to me.
OTOH 'constraint language' to specify various constraints
(which may be combined in arbitrary ways) could go quite
far. There is also time-honored approach of allowing "user"
extentions (possibly up to replacing "standard" scheduler).

Post by Don Y
Consider, however, that the service may be displaced by some higher
priority tasks executing on its hardware. So, it can't guarantee
that it ever *sees* the specific time you've specified; other
tasks may have preempted it. So, expecting to hit a specific
time is wishful thinking.

Again, this is question of design of your OS and your requirements.
If you have badly behaving third party code, then it may be hard to
give any warranty. But if you have need and apropriate resurces
then there are ways to get real-time behaviour (that is hitting
specified time window). Even if you do not have real-time
motivation, I think that time windows (as opposed to discrete
events) appear frequently. For example, you want to turn on
a device during night tariff.

Anyway, if you can not give reasonable warranties for time
when your scheduler runs, then it means that you do not care.
By extention, you do not care when scheduled tasks run.
Which may be fine for some purposes and in such case simplistic
handling which you propose may be good enough.

When you wrote your original post, I had someting different in
mind. Not so long ago I looked at one of home automation
system. I searched for some specific info and one of first
things that I found was a discussion about scheduling interface
in the system. A user wanted simply looking thing "if washing
machine is idle for some specified time, then turn its power off".
System allowed specifying conditions which caused some actions.
But there were limitations on what could be put as a condition,
and in effect correctly specifingt needed thing was rather
awkward.

Post by Don Y
Also, a specific time may not be appropriate for code written
some days/months/years earlier. For example, maybe I want to
ensure a task is activated when Daylight Savings Time ("Summer
Time") takes effect. So, that time could be hard-coded into
a task without regard for when the task was initially invoked.
[There are lots of fixed dates/times in our lives. So, these
could be unconditionally encoded without regard for *when*
that encoding took place]

Post by Waldek Hebisch
Concerning kernel, you should have in kernel enough to
support higher level code. In particular, if you have
"fast" tasks and care about their time performance, then
you may need real-time features in the kernel. If you do
not care, than usual general-purpose approach is enough.
But if you do not care, then why are you talking about
deadlines?

Because deadlines are always important. Even non-realtime
activities have some sense of timeliness.
I don't really care *when* the results of my checkbook balancing
program are available for inspection. But, I /probably/ would
like them before I write my next check! :>
Deadlines and value/importance let you (as a human) and
the machine decide where your efforts should be focused.
E.g., if you miss your spouse's birthday, should you still
attempt to recognize/celebrate it, even if late? What if
you miss your NEIGHBOR's birthday??

Post by Waldek Hebisch
Concerning tasks way in the past: this is very unusual case
for general purpose kernel. Namely, the only likely reason
for long delay is because system was turned off.

Note that I deliberately tried to address "distant past"
vs. "not so distant past". But, have left the magnitudes of
these differences unspecified.
"Not so distant past" seems like it is a condition that could
readily happen. E.g., if you tried to schedule an event but
found yourself delayed (by competing tasks) so that the scheduled
time was upon you before you got a chance to execute that function.
"Soooooo close!"
Given that this can happen, how "close" to missing that time
should you consider "close enough" that you can pretend you
managed to schedule it before the specified time. In which
case, one could argue that it should be activated "now"
(because "now" is really close to "then")

Post by Waldek Hebisch
But kernel
timers frequently are "volatile", that is they will be gone
when system is turned on again (if that is undesirable, then
higher level layer may restore timers that were supposed to
persist). AFAIK general purpose kernels handled timers
as you argue, that is dully activated corresponding task.
OTOH long delays were usually delegated to userspace which
could have complicated rules.

You have to assume these are unprivileged (out-of-kernel)
tasks -- because you don't want to bring more code *into*
the kernel.

Out-of-kernel does not mean unprivileged. Tasks may have
special rights which other tasks do not have. For example
one may allow elevated scheduling priority only to system
tasks. Even otherwise unprivileged task may be special
because it provides "well-known" interface to other tasks.

Post by Don Y
But, their importance is something you can't know, a priori
(you didn't write them!)
And, you can't determine if your chosen method of handling
"past" times will break the task that is making the request,
or not.
E.g., I can schedule a task to retrain my speech recognizer
in the wee hours of the morning (when the system load is lower),
based on input it has collected over the course of the day.
If I miss this time, it means that the quality of the recognizer
"tomorrow" will be less than it could have been. But, if the
retraining can happen TOMORROW night (again, in the system's
"off" time), then there isn't any significant loss.
OTOH, if I allow the retraining to happen at some delayed time that
interferes with other system tasks ("prime time"), other activities
could be compromised.
So, I argue that a simple criteria be adopted and the writer of
the task requesting the scheduling (and the task being scheduled)
address their own specific criteria IN EXPLICIT CODE to enforce
these, instead of hoping some other service can be made aware of
these requirements (by embelishing that service? by including
support fr up-calls to the task doing the scheduling??)
The "scheduler" can then just focus on ensuring each event gets
processed exactly once and at a time NO EARLIER than the time
specified. This is much easier for a developer to wrap his head
around than trying to second-guess the scheduler ("What if the
scheduler is blocked for some period of time? How will *it*
handle things when NOW is later than it *could* have been -- or,
when there are MANY competing things in the queue?")

For me important question is "who is the master?". Do you allow
third-party code to play any tricks it wants. Or do you give
control to the user. As a simple example, task may decide to
do some work at night, presumably to avoid interference with
user work. But a user may prefer to work at night, say on a
specific day. So user may wish to delay execution of the task.
If you want to empower users, then central facility is
proferable, as this is easier to control than separate tasks.

I have my own bias, I want computers to do what I want. For
expample, when I make a break I prefer to have some special
combination of keys to put system to sleep _immediately_.
When I work, I may think for longish time, and systems that
go to sleep based on "inactivity" distract me (especially
when I am trying to understand a text or a picture on the
screen and the screen goes blank).

[Don Y]

Post by Waldek Hebisch

Post by Don Y

Post by Waldek Hebisch
That is confusing formulation of the problem. First, what is
"the OS"?

Doesn't matter. The question is about how one should EXPECT an OS
to handle such things.

It matters. OS kernel should implement something simple, but enough
to support higher layers. At higher layers you may have something
more fancy.

You asked: "What OS?" The OS doesn't matter. Rather, how
the mechanism is modeled.

As you've said, the OS "should implement something simple".
Something easily conceptualized, even if it doesn't address
all possible use cases.

If, for example, the OS assumes that "past" events are no longer of
interest and ignores the request, then any higher layers have to
take that into account to COMPENSATE for the POLICY decision that
the OS has enforced.

Post by Waldek Hebisch

Post by Don Y

Post by Waldek Hebisch
If by "the OS" you mean operating system kernel, then you want
it as simple as possible and move various functions to upper
layers. In particular, in general purpose OS you may have
timers which activate a task some time after scheduled time
(hopefully close to scheduled time, but there is no warranty).
Maybe you are thinking of such a timer.

I am thinking one layer above that, in terms of abstraction.
Imagine some concurrent agency "watching the clock" and
taking action on your request to make something happen at a
particular time. *YOU* may be busy with something else.

I take this and what you write later that you think about
library interface for "applications".

A library function or a *service*.

Post by Waldek Hebisch

Post by Don Y

Post by Waldek Hebisch
However, there may be higher level "scheduler". For example
Unix 'cron' and 'at' are userspace programs, but are considered
part of operating system. 'cron' and 'at' are rather crude,
if you may need more. In particular, an automations system
typically must schedule various activities with specific time
constraints.
Anyway, your system probably is intended to do something useful
and that will lead to some requirements. Once you have more
concerte requirements, you can think how to best satisfy them.
If you have only handful of time-dependent tasks and each
differs a lot from other, then putting all higher level
time handling in the tasks may be reasonable. But usually
it pays to have common handling. As George mentioned there
can be interdependencies between task, so it is likely that
you may need a higher level scheduler to coordinate them.

My contention is that said service can't possibly know of all
of the interdependencies that could exist (examples provided
the time at which you desire the action to occur.

You can not solve _all_ scheduling problems. It does not

Correct.

Post by Waldek Hebisch
mean that you should not solve some. Windows style interface

You should only solve those that are commonplace -- and, only
as a convenience for your clients.

If nothing is common (unlike windows where applications may want
to know if the host is running on batteries, just came out of
hibernation, etc), then anything additional that you do in
the kernel just adds to the kernel's complexity without any
real reward.

Post by Waldek Hebisch
where you have a lot of specific cases and can not do anything
else than provided choices does not look attractive to me.
OTOH 'constraint language' to specify various constraints
(which may be combined in arbitrary ways) could go quite
far. There is also time-honored approach of allowing "user"
extentions (possibly up to replacing "standard" scheduler).

The "constraints" would then have to be available/accessible
to the scheduler. This leads to monolithic designs -- the scheduler
has to be able to know ANYTHING that may be of importance.

Instead, it should only have to know what is ESSENTIAL for it to perform
its action.

"Schedule this at 1:00AM but only if it did not rain, today"

Why should the scheduler have to know about local precipitation?
Instead, the task that thought that was a significant criteria
likely already knows about it; why not let *it* condition the
scheduler's actions?

Post by Waldek Hebisch

Post by Don Y
Consider, however, that the service may be displaced by some higher
priority tasks executing on its hardware. So, it can't guarantee
that it ever *sees* the specific time you've specified; other
tasks may have preempted it. So, expecting to hit a specific
time is wishful thinking.

Again, this is question of design of your OS and your requirements.
If you have badly behaving third party code, then it may be hard to
give any warranty.

The code doesn't even have to be malicious. I'm sure everyone
has had their workstation grind to a halt because something is
working as designed, but the set of applications aren't working
as EXPECTED.

(E.g., my browser routinely seems to run out of memory... or sockets...
or <whatever> meaning I can't open another page until I close it
and reopen it. Even if all the same tabs are reopened with it!)

Post by Waldek Hebisch
But if you have need and apropriate resurces
then there are ways to get real-time behaviour (that is hitting
specified time window). Even if you do not have real-time
motivation, I think that time windows (as opposed to discrete
events) appear frequently. For example, you want to turn on
a device during night tariff.

Yes. Or, the developer may have "noticed" (or expected)
certain behaviors that it can exploit. E.g., at this time,
the system load is low so it would be a good use of resources
(without incurring conflicts) for me to perform this resource
intensive task.

Post by Waldek Hebisch
Anyway, if you can not give reasonable warranties for time
when your scheduler runs, then it means that you do not care.

No. It means you care MORE about other things.

E.g., if power fails, I start shedding computational
loads so I can power down processors. The first to
go are those that are "unessential" or that can easily
be restarted to resume their state.

As I don't know how long a power outage will last
(but I *do* know how long my backup supply will support
a particular load set), I can't guarantee that "your"
task will be running anywhere when its deadline or
event times come to pass.

Similarly, if some high priority task manages to continually
preempt you, then you can't predict when you will next
"run" on the processor.

You've been preempted because your stated timing requirements
have been deemed insufficient to compete for the CURRENT
set of resources, given the other candidates vying for them.

The consequences of you not running may be serious. Or,
insignificant. Only YOU can determine that.

If, instead, you make a proclamation to the system
regarding your "importance"/timeliness, then it's easy
for foreign code to abuse that mechanism and always
claim to be "very important" and with "immediate"
timing criteria.

[To counter such abuse, if the workload manager sees
that it can't meet your requirements, it simply refuses
to run your application: "Why bother? You've already
CLAIMED to have these requirements and I *know* that
I can't meet them, so why forfeit resources trying to
meet your goals -- at the expense of other tasks -- only
to KNOW it will fail?"]

Post by Waldek Hebisch
By extention, you do not care when scheduled tasks run.
Which may be fine for some purposes and in such case simplistic
handling which you propose may be good enough.
When you wrote your original post, I had someting different in
mind. Not so long ago I looked at one of home automation
system. I searched for some specific info and one of first
things that I found was a discussion about scheduling interface
in the system. A user wanted simply looking thing "if washing
machine is idle for some specified time, then turn its power off".
System allowed specifying conditions which caused some actions.
But there were limitations on what could be put as a condition,
and in effect correctly specifingt needed thing was rather
awkward.

Exactly. Baking that into the OS is wrong. It should
implement MECHANISMS (i.e., the ability to load an application
into memory and ensure it's required resources are available)
and not POLICY (the criteria that an application may deem important)

Post by Waldek Hebisch

Post by Don Y

Post by Waldek Hebisch
But kernel
timers frequently are "volatile", that is they will be gone
when system is turned on again (if that is undesirable, then
higher level layer may restore timers that were supposed to
persist). AFAIK general purpose kernels handled timers
as you argue, that is dully activated corresponding task.
OTOH long delays were usually delegated to userspace which
could have complicated rules.

You have to assume these are unprivileged (out-of-kernel)
tasks -- because you don't want to bring more code *into*
the kernel.

Out-of-kernel does not mean unprivileged.

In *one* sense of the term. A service can operate at an
elevated sense of "importance" and timeliness. But, it
should be an appendage and not be able to interfere
with the operation of the kernel.

E.g., if a scheduler service dies, then NEW tasks can't
be invoked at specific times. But, tasks that are
already running -- or, explicitly loaded by them -- are
not affected. The 'damage" is contained.

One can then implement a "Lazarus service" that watches for
key services crashing and restarts them. E.g., there are
times when the display driver on my main workstation
crashes (usually coming out of hibernation or some other
significant hardware change). But, that is detected and
it is restarted so I am only momentarily inconvenienced.

Post by Waldek Hebisch
Tasks may have
special rights which other tasks do not have. For example
one may allow elevated scheduling priority only to system
tasks. Even otherwise unprivileged task may be special
because it provides "well-known" interface to other tasks.

Exactly. Everything in my world is capability based.
E.g., you can't even USE the scheduler if you haven't been
given a capability to do so.

"Why should the 'Flashlight' app in your phone be able to start
other tasks???"

Post by Waldek Hebisch

Post by Don Y
So, I argue that a simple criteria be adopted and the writer of
the task requesting the scheduling (and the task being scheduled)
address their own specific criteria IN EXPLICIT CODE to enforce
these, instead of hoping some other service can be made aware of
these requirements (by embelishing that service? by including
support fr up-calls to the task doing the scheduling??)
The "scheduler" can then just focus on ensuring each event gets
processed exactly once and at a time NO EARLIER than the time
specified. This is much easier for a developer to wrap his head
around than trying to second-guess the scheduler ("What if the
scheduler is blocked for some period of time? How will *it*
handle things when NOW is later than it *could* have been -- or,
when there are MANY competing things in the queue?")

For me important question is "who is the master?". Do you allow
third-party code to play any tricks it wants.

The app has to declare what it wasn't to be able to access
before it is installed in the system. If it attempts to
ask for some capabilities that it shouldn't need, then the
installer can refuse to allow the code to be installed.

If it misbehaves while running (abuses some capability
that it has been granted), the system can refuse to allow
it to be restarted.

Post by Waldek Hebisch
Or do you give
control to the user. As a simple example, task may decide to
do some work at night, presumably to avoid interference with
user work. But a user may prefer to work at night, say on a
specific day. So user may wish to delay execution of the task.
If you want to empower users, then central facility is
proferable, as this is easier to control than separate tasks.

Users can schedule some activities/actions through a "user accessible
service". If they want to schedule something at a "bad time",
that's their problem.

But, other tasks can have built-in heuristics that can provide hints
to when they should be scheduled IN THE ABSENCE OF SPECIFIC
CRITERIA FROM THE USER.

E.g., we irrigate at night. This is done because there are less
evaporative losses. And, because it is unlikely that users will
want to take showers at 3AM.

If the user overrides (cancels) such an action, the task
knows that the need for irrigation water still exists (just
because you don't want to irrigate now doesn't mean the
plants suddenly stop needing water). So, it tries to
find a better time.

Or, if the user explicitly gives it a better time (periodic
or aperiodic), then it learns from these preferences.

Post by Waldek Hebisch
I have my own bias, I want computers to do what I want. For
expample, when I make a break I prefer to have some special
combination of keys to put system to sleep _immediately_.

Yes. Hence the "SLEEP" option in the start menu.

Post by Waldek Hebisch
When I work, I may think for longish time, and systems that
go to sleep based on "inactivity" distract me (especially
when I am trying to understand a text or a picture on the
screen and the screen goes blank).

That requires adjusting the sleep interval. I often leave
a workstation for some other activity. I would be annoyed if
it DIDN'T go to sleep after I'd been "gone" (inactive) for 30
minutes.

OTOH, I've been scanning legal documents on my larger scanner.
It is networked so need not be located proximate to my
workstation. I can control it and retrieve scans without
being AT its side (though I need to be so to load new documents).

Often, I have to post-process a scan (e.g., if I have "receipts"
that I might want to scan alongside a document and, later,
"cut them off" as separate pages). While I am doing this,
the scanner often times out.

This is annoying as I have to restart the application.
(There is no obvious timeout value for the *application*,
though there is for the connection to the scanner).

But, tolerable. Compared to the time required to walk
over to the scanner (in another room to conserve on desk
space in the office), arrange the documents, walk back
to the workstation, perform a "preview" scan so I can
determine what portion of the flatbed is important, do
a real scan, and then import to photoshop, etc.

[A control in the scanning application that would let me
set the timeout -- or, disable it -- would be welcome]

[Waldek Hebisch]

Post by Don Y

Post by Waldek Hebisch

Post by Don Y

Post by Waldek Hebisch
That is confusing formulation of the problem. First, what is
"the OS"?

Doesn't matter. The question is about how one should EXPECT an OS
to handle such things.

It matters. OS kernel should implement something simple, but enough
to support higher layers. At higher layers you may have something
more fancy.

You asked: "What OS?" The OS doesn't matter. Rather, how
the mechanism is modeled.

I asked: what is "the OS"? In particular where you put the
line between OS and applications.

Post by Don Y
As you've said, the OS "should implement something simple".
Something easily conceptualized, even if it doesn't address
all possible use cases.
If, for example, the OS assumes that "past" events are no longer of
interest and ignores the request, then any higher layers have to
take that into account to COMPENSATE for the POLICY decision that
the OS has enforced.

Post by Waldek Hebisch

Post by Don Y

Post by Waldek Hebisch
If by "the OS" you mean operating system kernel, then you want
it as simple as possible and move various functions to upper
layers. In particular, in general purpose OS you may have
timers which activate a task some time after scheduled time
(hopefully close to scheduled time, but there is no warranty).
Maybe you are thinking of such a timer.

I am thinking one layer above that, in terms of abstraction.
Imagine some concurrent agency "watching the clock" and
taking action on your request to make something happen at a
particular time. *YOU* may be busy with something else.

I take this and what you write later that you think about
library interface for "applications".

A library function or a *service*.

You provide an interface. In modern times that is usually
in form of a function call. "Applications" need not care
if the actual implementation is library code or in kernel
or as a separate task.

Post by Don Y

Post by Waldek Hebisch

Post by Don Y

Post by Waldek Hebisch
However, there may be higher level "scheduler". For example
Unix 'cron' and 'at' are userspace programs, but are considered
part of operating system. 'cron' and 'at' are rather crude,
if you may need more. In particular, an automations system
typically must schedule various activities with specific time
constraints.
Anyway, your system probably is intended to do something useful
and that will lead to some requirements. Once you have more
concerte requirements, you can think how to best satisfy them.
If you have only handful of time-dependent tasks and each
differs a lot from other, then putting all higher level
time handling in the tasks may be reasonable. But usually
it pays to have common handling. As George mentioned there
can be interdependencies between task, so it is likely that
you may need a higher level scheduler to coordinate them.

My contention is that said service can't possibly know of all
of the interdependencies that could exist (examples provided
the time at which you desire the action to occur.

You can not solve _all_ scheduling problems. It does not

Correct.

Post by Waldek Hebisch
mean that you should not solve some. Windows style interface

You should only solve those that are commonplace -- and, only
as a convenience for your clients.
If nothing is common (unlike windows where applications may want
to know if the host is running on batteries, just came out of
hibernation, etc), then anything additional that you do in
the kernel just adds to the kernel's complexity without any
real reward.

As I wrote, IMO correct place for such a functionality is inside OS,
but outside the kernel.

The "nothing is common" part looks strange: it is pretty common
to want triggering on edges and also time windows.

Post by Don Y

Post by Waldek Hebisch
where you have a lot of specific cases and can not do anything
else than provided choices does not look attractive to me.
OTOH 'constraint language' to specify various constraints
(which may be combined in arbitrary ways) could go quite
far. There is also time-honored approach of allowing "user"
extentions (possibly up to replacing "standard" scheduler).

The "constraints" would then have to be available/accessible
to the scheduler. This leads to monolithic designs -- the scheduler
has to be able to know ANYTHING that may be of importance.
Instead, it should only have to know what is ESSENTIAL for it to perform
its action.
"Schedule this at 1:00AM but only if it did not rain, today"
Why should the scheduler have to know about local precipitation?
Instead, the task that thought that was a significant criteria
likely already knows about it; why not let *it* condition the
scheduler's actions?

In extendible scheduler that would be just abstract bit of information
and _application_ would tell its value to the scheduler. If there
are dependencies something have to know about them. In particular,
if you have dependence between independently developed application
it is natural to have scheduling as a third party.

Post by Don Y

Post by Waldek Hebisch

Post by Don Y
Consider, however, that the service may be displaced by some higher
priority tasks executing on its hardware. So, it can't guarantee
that it ever *sees* the specific time you've specified; other
tasks may have preempted it. So, expecting to hit a specific
time is wishful thinking.

Again, this is question of design of your OS and your requirements.
If you have badly behaving third party code, then it may be hard to
give any warranty.

The code doesn't even have to be malicious. I'm sure everyone
has had their workstation grind to a halt because something is
working as designed, but the set of applications aren't working
as EXPECTED.

I did not write "malicious". Rather, something like polling in
a busy loop instead of using blocking system call. Or Windows
graphic drivers which did not check that hardware is ready to
accept the data. Microsoft wrote that this was common because
it slightly improved average throughput of the driver. But
it could stall PCI bus, effectively freezing the whole computer
for say some miliseconds (IIUC now such access fails rather
quickly, but it used to be a problem).

Post by Don Y

Post by Waldek Hebisch
But if you have need and apropriate resurces
then there are ways to get real-time behaviour (that is hitting
specified time window). Even if you do not have real-time
motivation, I think that time windows (as opposed to discrete
events) appear frequently. For example, you want to turn on
a device during night tariff.

Yes. Or, the developer may have "noticed" (or expected)
certain behaviors that it can exploit. E.g., at this time,
the system load is low so it would be a good use of resources
(without incurring conflicts) for me to perform this resource
intensive task.

Post by Waldek Hebisch
Anyway, if you can not give reasonable warranties for time
when your scheduler runs, then it means that you do not care.

No. It means you care MORE about other things.

"Do not care", "do not care enough" or "care more about other
things" mean essentially the same here.

Post by Don Y
E.g., if power fails, I start shedding computational
loads so I can power down processors.

I wrote about "reasonable warranties". There is notion of
"force majeste" and clearly power failure counts as such.

Post by Don Y
The first to
go are those that are "unessential" or that can easily
be restarted to resume their state.
As I don't know how long a power outage will last
(but I *do* know how long my backup supply will support
a particular load set), I can't guarantee that "your"
task will be running anywhere when its deadline or
event times come to pass.
Similarly, if some high priority task manages to continually
preempt you, then you can't predict when you will next
"run" on the processor.

PLC-s have vendor provided I/O, so vendor knows how much time
I/O will take. They also have "logic", which is stright-line
code so vendor knows maximal execution time. Consequently,
if PLC task is given sufficiently high priority it will run
with predicable frequency. You may consider it unneeded,
you may think that other features are more important.
But in the end, either you care enough to provide something
like this or you do not care enough. In later case there
is no much point in thinking a lot about missing deadlines.

Post by Don Y
You've been preempted because your stated timing requirements
have been deemed insufficient to compete for the CURRENT
set of resources, given the other candidates vying for them.
The consequences of you not running may be serious. Or,
insignificant. Only YOU can determine that.
If, instead, you make a proclamation to the system
regarding your "importance"/timeliness, then it's easy
for foreign code to abuse that mechanism and always
claim to be "very important" and with "immediate"
timing criteria.
[To counter such abuse, if the workload manager sees
that it can't meet your requirements, it simply refuses
to run your application: "Why bother? You've already
CLAIMED to have these requirements and I *know* that
I can't meet them, so why forfeit resources trying to
meet your goals -- at the expense of other tasks -- only
to KNOW it will fail?"]

Post by Waldek Hebisch
By extention, you do not care when scheduled tasks run.
Which may be fine for some purposes and in such case simplistic
handling which you propose may be good enough.
When you wrote your original post, I had someting different in
mind. Not so long ago I looked at one of home automation
system. I searched for some specific info and one of first
things that I found was a discussion about scheduling interface
in the system. A user wanted simply looking thing "if washing
machine is idle for some specified time, then turn its power off".
System allowed specifying conditions which caused some actions.
But there were limitations on what could be put as a condition,
and in effect correctly specifingt needed thing was rather
awkward.

Exactly. Baking that into the OS is wrong. It should
implement MECHANISMS (i.e., the ability to load an application
into memory and ensure it's required resources are available)
and not POLICY (the criteria that an application may deem important)

I should have been clearer. The "home automation system" consisted
of devices (which ATM are of no interest) and user interface/scheduler
working as a normal application on standard OS. User interface
was supposed to be easy but allows users to define various
actions. The point is that in making it "easy" (and probably
simple to implement) user interface got crippled so that resonable
thing was hard to do.

[Don Y]

Post by Waldek Hebisch

Post by Don Y

Post by Waldek Hebisch

Post by Don Y

Post by Waldek Hebisch
That is confusing formulation of the problem. First, what is
"the OS"?

Doesn't matter. The question is about how one should EXPECT an OS
to handle such things.

It matters. OS kernel should implement something simple, but enough
to support higher layers. At higher layers you may have something
more fancy.

You asked: "What OS?" The OS doesn't matter. Rather, how
the mechanism is modeled.

I asked: what is "the OS"? In particular where you put the
line between OS and applications.

Sorry, I interpreted that as *WHICH* OS.

My OS exports many functions to userland in an effort to keep
the TCB small. Hence the desire to constrain the "abilities"
(responsibilities?) of anything that has to run IN the TCB

Post by Waldek Hebisch

Post by Don Y

Post by Waldek Hebisch

Post by Don Y

Post by Waldek Hebisch
If by "the OS" you mean operating system kernel, then you want
it as simple as possible and move various functions to upper
layers. In particular, in general purpose OS you may have
timers which activate a task some time after scheduled time
(hopefully close to scheduled time, but there is no warranty).
Maybe you are thinking of such a timer.

I am thinking one layer above that, in terms of abstraction.
Imagine some concurrent agency "watching the clock" and
taking action on your request to make something happen at a
particular time. *YOU* may be busy with something else.

I take this and what you write later that you think about
library interface for "applications".

A library function or a *service*.

You provide an interface. In modern times that is usually
in form of a function call. "Applications" need not care
if the actual implementation is library code or in kernel
or as a separate task.

Yes. But the *implementation* cares. Everything that resides
in the TCB represents a potential threat to the TCB. If you
can implement that functionality OUTSIDE the TCB, the application
doesn't need to know how or why.

In the absence of that "scheduling service", there is nothing to
prevent an APPLICATION to develop one -- for use by other
applications or its own particular needs. There is nothing
about it that dictates it need be part of the OS. Or the TCB.

[Does every device need to support a filesystem? Does filesystem
support need to reside IN the TCB??]

Post by Waldek Hebisch

Post by Don Y
You should only solve those that are commonplace -- and, only
as a convenience for your clients.
If nothing is common (unlike windows where applications may want
to know if the host is running on batteries, just came out of
hibernation, etc), then anything additional that you do in
the kernel just adds to the kernel's complexity without any
real reward.

As I wrote, IMO correct place for such a functionality is inside OS,
but outside the kernel.
The "nothing is common" part looks strange: it is pretty common
to want triggering on edges and also time windows.

Only if you can make those guarantees. Hence the original
question: what do you do when the world conspires to make
it such that your scheduling *request* doesn't execute
until AFTER the desired time has occurred? Whether a window
or a point-in-time, it is possible (outlined in example)
that this could be in YOUR past.

Ignore the request?
Act on it "immediately" (which is defined as whenever the
scheduler gets a CHANCE to act on it)?
Return an error (rely on the task making the request to resubmit)?

Post by Waldek Hebisch

Post by Don Y
Instead, it should only have to know what is ESSENTIAL for it to perform
its action.
"Schedule this at 1:00AM but only if it did not rain, today"
Why should the scheduler have to know about local precipitation?
Instead, the task that thought that was a significant criteria
likely already knows about it; why not let *it* condition the
scheduler's actions?

In extendible scheduler that would be just abstract bit of information
and _application_ would tell its value to the scheduler. If there

But that "information" pertains to some quantity, quality, other event.
E.g., using JUST time means the scheduler only needs to know about
*time*. It doesn't need to know about (e.g.) precipitation.

Post by Waldek Hebisch
are dependencies something have to know about them. In particular,
if you have dependence between independently developed application
it is natural to have scheduling as a third party.

Post by Don Y

Post by Waldek Hebisch

Post by Don Y
Consider, however, that the service may be displaced by some higher
priority tasks executing on its hardware. So, it can't guarantee
that it ever *sees* the specific time you've specified; other
tasks may have preempted it. So, expecting to hit a specific
time is wishful thinking.

Again, this is question of design of your OS and your requirements.
If you have badly behaving third party code, then it may be hard to
give any warranty.

The code doesn't even have to be malicious. I'm sure everyone
has had their workstation grind to a halt because something is
working as designed, but the set of applications aren't working
as EXPECTED.

I did not write "malicious".

But you (I) have to address that possibility. Or, take the Apple
"walled garden" approach where you only allow code to run that
YOU have reviewed and "blessed".

Post by Waldek Hebisch
Rather, something like polling in
a busy loop instead of using blocking system call. Or Windows

There's no practical difference between that and running
an endless malloc/free loop or requesting resources that
others might need just "out of spite"

Post by Waldek Hebisch
graphic drivers which did not check that hardware is ready to

I move the drivers to separate user-land tasks. Driver misbehaves
(malicious/stupid/buggy) and the driver suffers. It's silly
to be compromising code quality in the 21st century for
small performance improvements.

Post by Waldek Hebisch
accept the data. Microsoft wrote that this was common because
it slightly improved average throughput of the driver. But
it could stall PCI bus, effectively freezing the whole computer
for say some miliseconds (IIUC now such access fails rather
quickly, but it used to be a problem).

Post by Don Y

Post by Waldek Hebisch
But if you have need and apropriate resurces
then there are ways to get real-time behaviour (that is hitting
specified time window). Even if you do not have real-time
motivation, I think that time windows (as opposed to discrete
events) appear frequently. For example, you want to turn on
a device during night tariff.

Yes. Or, the developer may have "noticed" (or expected)
certain behaviors that it can exploit. E.g., at this time,
the system load is low so it would be a good use of resources
(without incurring conflicts) for me to perform this resource
intensive task.

Post by Waldek Hebisch
Anyway, if you can not give reasonable warranties for time
when your scheduler runs, then it means that you do not care.

No. It means you care MORE about other things.

"Do not care", "do not care enough" or "care more about other
things" mean essentially the same here.

No. You are still making progress -- just not on the
particular task you're looking at. Who is to say that
it's actions can't happen later?

If you have an open system, then you can't guarantee
any particular set of resources are available. E.g.,
how does your windows machine schedule a task for
4AM -- if you've powered it off at 3AM?

If you wanted such an ability, the OS (likely) would have
to export a "power on service" -- which would likely be
privileged -- to certain applications. Otherwise, a
bug or malicious actor could power your system on
even when doing so would be deleterious to it.

Post by Waldek Hebisch

Post by Don Y
E.g., if power fails, I start shedding computational
loads so I can power down processors.

I wrote about "reasonable warranties". There is notion of
"force majeste" and clearly power failure counts as such.

That;s just an easy.extreme example. What if some other
task -- deemed more important than you -- hogs the resources
(not just CPU time) that YOU need. As far as YOU are concerned,
the processor is OFF -- because none of your code is executing.

And, in an open system, there are no limits on how long this
condition can persist. (my browser routinely eats up CPU
cycles while "not responding" -- GC?) If I wanted to use
*it* to do something during these periods, I'm SoL.

Post by Waldek Hebisch

Post by Don Y
The first to
go are those that are "unessential" or that can easily
be restarted to resume their state.
As I don't know how long a power outage will last
(but I *do* know how long my backup supply will support
a particular load set), I can't guarantee that "your"
task will be running anywhere when its deadline or
event times come to pass.
Similarly, if some high priority task manages to continually
preempt you, then you can't predict when you will next
"run" on the processor.

PLC-s have vendor provided I/O, so vendor knows how much time
I/O will take. They also have "logic", which is stright-line
code so vendor knows maximal execution time. Consequently,
if PLC task is given sufficiently high priority it will run
with predicable frequency. You may consider it unneeded,
you may think that other features are more important.
But in the end, either you care enough to provide something
like this or you do not care enough. In later case there
is no much point in thinking a lot about missing deadlines.

You provide mechanism, not policy. You let the USER -- by
his choice of applications to install, etc. -- decide on the
policy that is appropriate for him.

We don't irrigate when we want to take showers. Because the
irrigation system uses a LOT of water and that causes a
noticeable drop in water pressure.

As a kid, we wouldn't use hot water in the kitchen while
someone was showering -- because the water heater couldn't
keep up with the demand.

ONE solution is to increase the resources available (a
larger water main, a larger water heater). Another is
to alter your usage to fit within the resources available.
Disable the irrigation system if you MUST take a shower, NOW.

Guarantees are only possible in closed systems.

Military, with its ENORMOUS budget, can only fend off
a particular number of incoming projectiles before its
systems are overwhelmed.

Should it simply "give up" when that condition (which is under
the control of its ADVERSARIES) exists? Or, should it prioritize
the targets that it defends??

I can start more processes than this PC can support (with
"reasonable" response times). Should the PC prevent me from
doing so? Or, should it make its best effort at trying to
meet my needs and let ME decide how to cope with "overload"?

The machine should provide the mechanisms and the user (or
application) should determine the POLICY.

Post by Waldek Hebisch

Post by Don Y

Post by Waldek Hebisch
By extention, you do not care when scheduled tasks run.
Which may be fine for some purposes and in such case simplistic
handling which you propose may be good enough.
When you wrote your original post, I had someting different in
mind. Not so long ago I looked at one of home automation
system. I searched for some specific info and one of first
things that I found was a discussion about scheduling interface
in the system. A user wanted simply looking thing "if washing
machine is idle for some specified time, then turn its power off".
System allowed specifying conditions which caused some actions.
But there were limitations on what could be put as a condition,
and in effect correctly specifingt needed thing was rather
awkward.

Exactly. Baking that into the OS is wrong. It should
implement MECHANISMS (i.e., the ability to load an application
into memory and ensure it's required resources are available)
and not POLICY (the criteria that an application may deem important)

I should have been clearer. The "home automation system" consisted
of devices (which ATM are of no interest) and user interface/scheduler
working as a normal application on standard OS. User interface
was supposed to be easy but allows users to define various
actions. The point is that in making it "easy" (and probably
simple to implement) user interface got crippled so that resonable
thing was hard to do.

That's an opportunity for a supplier to offer an "assistant"
(if they don't control the actual system) to assist the user.
Eventually, the original developer will come to realize
THEY should offer the assistant's functionality in the base
product. But, that only needs to be at some level of
abstraction between the user and the hardware. It need not
be part of the OS itself (even if the user THINKS of it as
part of the "OS")

[Don Y]

Post by Don Y

I should have been clearer.  The "home automation system" consisted
of devices (which ATM are of no interest) and user interface/scheduler
working as a normal application on standard OS.  User interface
was supposed to be easy but allows users to define various
actions.  The point is that in making it "easy" (and probably
simple to implement) user interface got crippled so that resonable
thing was hard to do.

That's an opportunity for a supplier to offer an "assistant"
(if they don't control the actual system) to assist the user.
Eventually, the original developer will come to realize
THEY should offer the assistant's functionality in the base
product.  But, that only needs to be at some level of
abstraction between the user and the hardware.  It need not
be part of the OS itself (even if the user THINKS of it as
part of the "OS")

I.e., the "original system" offers the mechanisms (to talk to the
various "devices") but the policy about how to use those mechanisms
is embodied in the "assistant".

[Of course, if the system is (completely) closed, then the user is stuck
with whatever assumptions the developer baked into the application]

[Waldek Hebisch]

Post by Don Y

Post by Don Y

I should have been clearer.  The "home automation system" consisted
of devices (which ATM are of no interest) and user interface/scheduler
working as a normal application on standard OS.  User interface
was supposed to be easy but allows users to define various
actions.  The point is that in making it "easy" (and probably
simple to implement) user interface got crippled so that resonable
thing was hard to do.

That's an opportunity for a supplier to offer an "assistant"
(if they don't control the actual system) to assist the user.
Eventually, the original developer will come to realize
THEY should offer the assistant's functionality in the base
product.  But, that only needs to be at some level of
abstraction between the user and the hardware.  It need not
be part of the OS itself (even if the user THINKS of it as
part of the "OS")

I.e., the "original system" offers the mechanisms (to talk to the
various "devices") but the policy about how to use those mechanisms
is embodied in the "assistant".
[Of course, if the system is (completely) closed, then the user is stuck
with whatever assumptions the developer baked into the application]

AFAIK the system was open-source, so theoreticaly third parties could
add any improvements they wished. But it seems that original
developer considered UI as major added-value and there were no
official/documented way to decouple UI from other parts (there were
documented way to add new devices).

There is also question of project dynamics, there are competing
projects and this one probably did not attract much interest
among outside developers. And of course software evolves,
so this could be solved in the future.

[Don Y]

Post by Waldek Hebisch

Post by Don Y

Post by Don Y

I should have been clearer.  The "home automation system" consisted
of devices (which ATM are of no interest) and user interface/scheduler
working as a normal application on standard OS.  User interface
was supposed to be easy but allows users to define various
actions.  The point is that in making it "easy" (and probably
simple to implement) user interface got crippled so that resonable
thing was hard to do.

That's an opportunity for a supplier to offer an "assistant"
(if they don't control the actual system) to assist the user.
Eventually, the original developer will come to realize
THEY should offer the assistant's functionality in the base
product.  But, that only needs to be at some level of
abstraction between the user and the hardware.  It need not
be part of the OS itself (even if the user THINKS of it as
part of the "OS")

I.e., the "original system" offers the mechanisms (to talk to the
various "devices") but the policy about how to use those mechanisms
is embodied in the "assistant".
[Of course, if the system is (completely) closed, then the user is stuck
with whatever assumptions the developer baked into the application]

AFAIK the system was open-source, so theoreticaly third parties could
add any improvements they wished. But it seems that original
developer considered UI as major added-value and there were no
official/documented way to decouple UI from other parts (there were
documented way to add new devices).

Obviously (?) the primary developer(s) had a different priority
than BEING open source. I.e., they still wanted control over
the project/product and thought keeping parts of it "closed"
(effectively) gave them that.

Post by Waldek Hebisch
There is also question of project dynamics, there are competing
projects and this one probably did not attract much interest
among outside developers. And of course software evolves,
so this could be solved in the future.

I take a different approach. I'm not interested in "controlling"
a project -- any more than I am interested in babysitting
someone's children!

Nor am I particularly interested in "making money"; I've done enough
of that in my career to be "comfortable" (and have no one who depends
upon me for some "inheritance"). Having "customers" is like having
hemorrhoids!

Rather, I want to see my designs USED -- even if they are transformed
or repurposed in the process.

I've chosen to tackle a few problems that I (personally) consider as
important in my prototype:
- addressing users with varying "disabilities" (to be non-politically correct)
instead of unilaterally relying on sight and mobility
- addressing users who might otherwise "need" to move into "assisted
living" facilities simply because they don't have the time or
faculties to remember all the "little things" that are important
if you continue to occupy your home

[E.g., You left something on the stove. The toilet mechanism is leaking;
call a plumber. The garage door is still open, an hour after you've returned
home. The freezer is no longer maintaining the desired temperature, your
foods are likely going to spoil. The roses need to be watered. The "emitter"
for the citrus trees appears to be clogged as water isn't flowing thru it
at the rate it should be. You are *lying* on the bathroom floor. You need
to purchase more coffee beans. You've been going to the bathroom excessively;
perhaps you need to consult a doctor? You forgot to take your medications.
The wheelchair needs to be recharged. etc.]