# -*- coding: latin-1 -*-
"""

   ToDoExtension.py                   Todo-Extension for WikidPad

   Written by Christian Ziemski

   For some more documentation and new versions have a look at http://www.ziemski.net/wikidpad/todo_extension.html

History:
--------

  2006-10-30 first version
  2006-11-02 bug fixes
  2006-11-09 Fix to work with pages named like [no wiki word] as well  (beta)
  2006-11-10 Fix to work with pages named like [no WikiWord] as well  (beta)
  2006-11-11 Use internal function to check WikiWord
  2006-11-16 several small mods (config, filter, color)
  2007-01-31 fixes to run in WikidPad up to 1.8 and in the new 1.9 as well
  2007-06-16 fixes tags with sub-text and date; text without trailing LF at ToDo page
  2007-06-18 fix for problem with multiple-tabs (todos on wrong page inserted)
  2007-07-27 fix for displayMessage in 1.9 (up to 1.9beta7)
  2007-07-27 above fix removed because no longer necessary due to 1.9beta8
  2007-07-31 if (onlyRealTodos == 0): that other category (like done, question ...) is shown in list
  2007-08-03 only a fix for above
  2007-09-08 new config option extendedLinks for links with search capability (only in editor window(?)) (test for Christopher)
  2007-10-02 new config: setBookmark added // list of todos sorted within every tag block
  2007-10-09 new config added: realDatesLast for sorting and infoPopupTimeDelta for suppressing frequent popups
  2007-10-11 time calculation changed a bit (tip from Jouni)
  2007-10-17 display weekday for dates // optional split of real-date-todos and normal ones into separate pages (wish from Jouni)
  2007-10-19 little config changes, some cleaning up
  2007-10-20 fixes // display week number for dates
  2007-10-22 display possible tags in entries on ToDoCalendar page too
  2007-12-06 fix for a bug if onlyRealTodos==True and todos with embedded ':' in their text
  2008-01-22 some checking for valid dates
  2008-02-20 Begin of rewrite, thanks to Marko Mahnic.
  2008-02-23 new outDateFormat configuration option
  2008-02-24 fix: new outDateFormat configuration option doesn't work on Windows automatically. Fallback implemented.
  2008-02-24 added a configuration file ToDoExtension.cfg for local changes
  2008-11-23 added new date formats "*-Wnn" for 'Monday of every week nn, every year' and "*-W*-d" for 'every d weekday (Monday=1, Tuesday=2...)'; fixes in date format CWnn and KWnn (suggested by Jouni Miettunen)
  2009-01-24 fix: optional filter for todo page (changed configuration option) / add: config option for deleting attributes from todo entry (suggested by Kurt Woodham)
  2009-02-02 Works with WikidPad 2.0alpha too now. Some fixes (e.g. bug when realDatesSeparate=True)
  2009-02-03 Better error message for invalid dates, but still needs more work
  2009-02-05 removed unused srePersistent / modified version check / new config option: showVersion

Why all that?
-------------

I was looking for a good To-Do/Notes/PIM application for a long time and found WikidPad.

Loosely based on the following extensions I wrote my own todo-extension..

    http://wikidpad.python-hosting.com/wiki/GettingThingsDone
        The starting point. NextActions by Eric Neumann
        Enhanced by icosahedral to organize by context instead of by project name.

    http://wikidpad.python-hosting.com/wiki/SortedTodos
        JaysenNaidoo version to sort the page according to tags  (todo.<tag>:)
        Mike Crowe added dynamic tags

I added date calculations, date sorting, a separate calendar page and a configuration section.
(The formulae for date calculation from week number I found somewhere in 'the internet'.)


Description:
------------

If a wikiWord beginning with "ToDo" is created or selected (viewed) it automatically
collects all 'todo' entries from all other pages and inserts them here (after a placemark).
Of course you can write own text before that placemark without being changed.

If the wikiWord has additional characters after the "ToDo" like "ToDoPrivate" those characters
"Private" are used as filter (case insensitive).
If there are spaces or underlines between "ToDo" and the following characters (e.g "ToDo_Private")
these are ignored and so still the filter is "Private". (This may be better readable.)
Only Todos containing that filter string (in its name or in the full entry) are collected.
(Behavior depends on configuration option "filterTodos" below.)

Note: If the optional Calendar feature is enabled below, it is "ToDoCalendar" and "ToDoCalendarPrivate" in the example above.)
 

The todos are sorted by 'tag' the following way:

    1.) Completely untagged                                e.g. 'todo: shopping'

    2.) Tagged with special tags.                          e.g. 'todo.High: pay taxes'
          Sort order and display see config section below

    3.) Tagged with a date (at End-Of-Tag!)
          Sorted by date (earliest first in list)
          Marked as 'GONE', 'TODAY', 'n hours left', 'n days left' dependent on dayrange below
          Displayed in different formats, dependent the same way.

          The following date formats are recognized:
              "yyyy-mm-dd"   # standard date                               e.g. 'todo.2006-10-31: Halloween 2006'
              "mm/dd/yyyy"   # GB/US date
              "dd.mm.yyyy"   # European date
              "yyyy-Wnn-d"   # a week with the DayOfWeek d (Mon(1)-Sun(7)) e.g. todo.2007-W03-2:  Tuesday of 3rd week in 2007
              "yyyy-Wnn"     # a week in a given year (Monday)             e.g. todo.2008-W12:    Week 12 in 2008
              "*-W*-d"       # weekday d (Mo(1)-Su(7)) in every week+year  e.g. todo.*-W*-1:      go to work on Monday
              "*-Wnn"        # given week, every year (Monday)             e.g. todo.*-W52:       prepare end of year
              "CWnn"         #   same as above: Calendar Week
              "KWnn"         #   same as above: Calendar Week (German: Kalender Woche)
              "*-mm-dd"      # a date every year                           e.g. todo.*-10-31:     Halloween, in every year
              "*-*-dd"       # a day every month, every year               e.g. todo.*-*-01:      Every month, the 1st day
              "*-*-*"        # every day

          The dates with "*" are optimistic ones: if such a date is in the past
          it is assumed as "for next week/month/year" and not as "GONE"!

          If there are subtags like the string "Peter" in "todo.Peter.2008-09-16: Birthday" the subtag is listed in parentheses on the todo page.


    4.) otherwise tagged                                       e.g. 'todo.family: holidays'


The following WikiDocumentAttributes will be added to that generated page
(If there isn't already a custom [icon: ...] set, see configuration below):
    [icon: spanner]
    [color: magenta]
    [bookmarked=true]


Note: Within the WikidPad help wiki this hook/extension does nothing.
"""

# todo: Description of new Installation. No longer in WikidPadHooks.py, but here in TodoExtension.py
# todo: Modify docu on Website (due to "self." in config section and new installation)

#========================================================================================

version = "2009-02-05"

WIKIDPAD_PLUGIN = (("hooks", 1),)

# Only a single hook for this original function from WikidPadHooks.py is installed herein.

def openedWikiWord(docPagePresenter, wikiWord):
    """
    Called when a new or existing wiki word was opened successfully.

    wikiWord -- name of the wiki word to create
    """

    if (wikiWord[:4] == "ToDo"):

        if wpversion < "1.9":
            wikiname = docPagePresenter.wikiName
        else: 
            wikiname = docPagePresenter.getMainControl().wikiName 

        # to avoid modification of help-wiki
        if wikiname == "WikidPadHelp":
            return

        todo = Todo(docPagePresenter)  # Initialize

        todo.ProcessTodoPage(wikiWord) # do the work


#========================================================================================

CFG_FILE="ToDoExtension.cfg"

import os, sys
import locale
from re       import compile, match
from time     import strftime, strptime, mktime, localtime, time
from datetime import datetime, timedelta, date
import wx
#import sys   # at least for debugging # todo: create log/debug function

# Get WikidPad's version. More infos in Consts.py
#   VERSION_TUPLE = ("wikidPad", 2, 0, 1, 4)
#   VERSION_STRING = "wikidPad 2.0alpha01_04"
try:
  from WikidPadStarter import VERSION_TUPLE
  (vbr,vmaj,vmin,vsam,vpat) = VERSION_TUPLE
  wpversion = "%d.%d" % (vmaj,vmin) 
except:
  # Attention: This determination of the version is not foolproof!
  # Coded this way to remain compatible back to WikidPad 1.8
  from WikidPadStarter import VERSION_STRING        # e.g. "wikidPad 1.8beta6", "wikidPad 2.0alpha01_03"
  wpversion = VERSION_STRING[9:12]                    # e.g. "1.8"

#--- functions ----------------------------------------------------------------------

def LeapYr(y):
    if (y % 4):
        return False;
    if (y % 100):
        return True
    return not(y % 400)

def Jan1DoWG(y):
    y = y - 1
    return (int(1.25 * y) - int(y / 100) + int(y / 400) + 1) % 7

def DoYLtoMonth(J,L):
    if (J < 3):
       return 1
    return 1 + int((303 + 5 * (J - 59 - L)) / 153)

def DoYMLtoDate(J,M,L):
    if (M < 3):
        return J - (31 * (M - 1))
    return J - (int((153 * M - 2) / 5) - 32 + L)

def RevWN(Y,W,D):
    Yr = Y
    Leap = LeapYr(Yr)

    X = (Jan1DoWG(Yr) + 2) % 7
    J = 7 * W + D - X - 4
    Mth = int(DoYLtoMonth(J, Leap))
    Dte = int(DoYMLtoDate(J, Mth, Leap))

    if (Mth == 13):
        Yr = Yr + 1
        Mth = 1
    if (Dte < 1):
        Yr = Yr - 1
        Mth = 12
        Dte = Dte + 31
    return [Yr, Mth, Dte]

#--- end of functions ---------------------------------------------------------------

class Todo:
    def __init__(self, wikidpad):
        self.wikidpad = wikidpad
        self.config_set_defaults()
        cfg_file=os.path.join(os.path.dirname(os.path.realpath(os.sys.argv[0])), "user_extensions", CFG_FILE)
        #todo: easier? perhaps the current path is stored somewhere already?!
        self.config_read(cfg_file)

    def config_read(self, cfg_file):
        if os.path.exists(cfg_file):
            #try:                              #todo: better error handling!!!  
            cfg=open(cfg_file)
            config=cfg.read()
            cfg.close()
            try:
                exec config                        #todo: choose a more secure way in favour of exec! And better error handling!
            except:
                self.wikidpad.displayMessage("Error!", "Syntax error in configuration file:\n\n" + cfg_file + "\n\n" + str(sys.exc_info()[1]))

            self.zerror=False
            #except:
            #    self.zerror=True
        else:
            self.zerror=True
        pass

    def config_set_defaults(self):
        """
        Default configuration values
        ----------------------------

        ATTENTION:
        Don't change them here since they will be overwritten by the next update of this program.
        Instead maintain your own local configuration in file "user_extensions/ToDoExtension.cfg".

        Simply copy one or more config values from here into that file and modify them to your needs.
        Be careful to let them begin at BeginOfLine: the "self.<...>" mustn't be indented there!
        I'll try to find a better solution ...
        """

        #------------------------------------------------------------------------------------------
        # tags contain the (Tag, TagHeader, '') triples
        # TagHeaders are the descriptive headings that will be shown for each category.
        # UNTAGGED is a special one (mandantory!) to collect the untagged "todo:"

        self.tags = [
                ('UNTAGGED',     '++++ Not yet tagged', '', ''),
                ('High',         '+++  HIGH!', '', ''),
                ('Next',         '++++ Next Actions', '', ''),
                ('ThisWeek',     '++++ This Week', '', ''),
                ('SomeDay',      '++++ SomeDay / Maybe', '', ''),
                ('TimeToTime',   '++++ From time to time', '', ''),
                ('Low',          '++++ Tagged as LOW', '', ''),
                ('VeryLow',      '++++ Tagged as Very LOW', '', ''),
               ]

        #todo: make sure that UNTAGGED is always there!

        self.placemark  = "++++ ________auto-collected todos________"
        self.underline  = "_______"
        self.spacerline = "-" * 4
        self.bullet     = "   * "

        self.iconString        = '[icon: spanner]'

        self.colorStringNormal = '[color: black]'       # Colors for the ToDo entry in the tree
        self.colorStringNext   = '[color: orange]'      #
        self.colorStringToday  = '[color: red]'         #
        self.colorStringMissed = '[color: magenta]'     #

        self.notagHeading = "++++ "               # for unknown tagged entries
        self.dateHeading  = "++++ "               # for date entries far away (distance see below)
        self.dateHeadingX = "+++  "               # for date entries in near future or missed in past

        self.missedString = " ___ (+GONE+)"
        self.nowString    = " ___ !!! TODAY !!!"

        self.outDateFormat = ""         # if this is set = "" the local date format is used, else set it to "%m/%d/%Y", "%d.%m.%Y" or "%Y-%m-%d" for example
                                        # Note: On Windows the determination of the locale doesn't work yet. So you have to set it manually here.
                                        #       Or it defaults to "%Y-%m-%d".
                                        # Please use only %m, %d  for month and day  and %y, %Y for 2-/4-digit year
                                        # (Otherwise the optional separation of date driven todos may fail.)

        self.showWeekday   = True       # show weekday after dates
        self.weekdayFormat = " (%s,"    # %s is replaced by the weekday as in the array below,
                                        # together with the optional week number (see below) it gives e.g. " (Fri, W42)"

        self.weekdays      = ['Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat', 'Sun']  # English
        #self.weekdays     = ['Mo', 'Tu', 'We', 'Th', 'Fr', 'Sa', 'Su']         # English
        #self.weekdays     = ['Mo', 'Di', 'Mi', 'Do', 'Fr', 'Sa', 'So']         # German

        self.showWeeknumber   = True      # show weeknumber after dates
        self.weeknumberFormat = " W%02d)" # %02d is replaced by the weeknumber (2 digits with leading 0-padded)

        ##old: self.checkSearchString (up to 2009-01-23)
        self.filterTodos = 1            # 0 = no filtering: ignore characters after 'ToDo'
                                        # 1 = case insensitive filter string after 'ToDo', only in todo's name
                                        # 2 = case insensitive filter string after 'ToDo', in complete todo entry (line)
                                        ##no longer valid as of 2009-01-24## 2 = case sensitive filter string after 'ToDo'

        self.deleteAttributes = 1       # 0 = let possible attributes [...:...] in a todo entry alone
                                        # 1 = delete possible attributes [...:...] from a todo entry to avoid multiple attr errors

        self.dayrange = 7               # to mark date entries in that range as "more important"
                                        # than the ones in "far" future

        self.showInfoPopup      = True  # about todos for today / missed ones
        self.infoPopupTimeDelta = 2     # number of minutes where no new popup is shown after last visit of ToDo page

        self.onlyRealTodos = True       # True = ignore done, wait, question etc.
                                        # False = all from wikidPads todo-like category

        self.setBookmark = True         # set the ToDo page [bookmarked=true] so it is listed in the Bookmark popup (Shift-Ctrl-B)

        self.showVersion = False        # show version of WikidPad and extension (mainly for debugging)

        self.realDatesLast = False      # False: sorting is: untagged - known tags - dates - newly tagged  (the default)
                                        # True:  sorting is: untagged - known tags - newly tagged - dates  (test for Jouni)

        self.realDatesSeparate = False  # False: all todos in one page: ToDo
                                        # True:  todos with real dates on a page "ToDoCalendar", all others on the normal page "ToDo"

        # The following feature works, but doesn't look nice, really! So better let it disabled...
        self.extendedLinks = False      # True  = create links with search capability on target page (only working in editor window!?)
                                        # False = create normal links


    def delAttribs(self, s):
        sattr = compile("\[[^:]+? *: *[^:]+?\]")
        result = sattr.search(s)
        while result:
            s = s[0:result.start()] + s[result.end():]
            result = sattr.search(s)
        return s

    def ProcessTodoPage(self, wikiWord):
        global wpversion

        missedTodos      = 0
        todosForToday    = 0
        todosForNextDays = 0

        colorStringFlag  = 1     # set color string per default, checked later below

        myNow     = strftime("%Y-%m-%d %H:%M", localtime(time()))
        myToday   = strftime("%Y-%m-%d", localtime(time()))
        thisweek  = int(strftime("%W", localtime(time()))) + 1
        warndate  = strftime("%Y-%m-%d", localtime(time() + self.dayrange*24*60*60))
        (thisyear,thismonth,thisday,thishour,thisminute,thisecond,thiswday,thisdoy,thisdst) = localtime(time())

        if self.outDateFormat == "":
            try:
                locale.setlocale(locale.LC_ALL, '')
                self.outDateFormat = locale.nl_langinfo(locale.D_FMT)
            except:
                self.outDateFormat = "%Y-%m-%d"

        if wpversion < "1.9":
            todosFull = self.wikidpad.wikiData.getTodos()
            editor    = self.wikidpad.getActiveEditor()
        else:
            todosFull = self.wikidpad.getMainControl().wikiData.getTodos()
            editor    = self.wikidpad.getSubControl("textedit")

        if wpversion <= "1.9":
            langHelper = self.wikidpad.getFormatting()
        else:
            # Retrieve internal name of current wiki language
            wikiLang = self.wikidpad.getWikiDocument().getWikiDefaultWikiLanguage()
            # Get language helper (an instance of WikidPadParser._TheHelper)
            langHelper = wx.GetApp().createWikiLanguageHelper(wikiLang)
            # For the newly published 2.0alpha01_03 it is then:
            #langHelper.isCcWikiWord(...)

        srchstr = wikiWord[4:]

        isCalendar = False
        if srchstr.startswith("Calendar"):
            srchstr = srchstr[8:]          # remove that "Calendar" from srchstr to get the real one
            isCalendar = True

        srchstr = srchstr.strip(" _")      # remove possible leading/trailing spaces and underscores from srchstr

        # optionally filter the todos
        if self.filterTodos == 0:
            todos = [todo for todo in todosFull]                                            # without checking srchstr
        elif self.filterTodos == 1:
            todos = [todo for todo in todosFull if (srchstr.upper() in todo[1][4:todo[1].find(":")].upper()) ]   # case insensitive search in todo's name
        elif self.filterTodos == 2:
            todos = [todo for todo in todosFull if (srchstr.upper() in todo[1].upper()) ]   # case insensitive in complete todo entry (the todo line)
        
        if self.onlyRealTodos:
            todos = [todo for todo in todos if todo[1][0:4] == "todo" ]


        # prepare text page to insert the collected todos after placemark

        tscheck = compile("(@ [0-9]{4}-[0-1][0-9]-[0-3][0-9] [0-2][0-9]:[0-5][0-9])")   # (@ 2007-10-28 22:09)

        st = editor.FindText(0, editor.GetLength(), self.placemark, 0)

        if st != -1:                                    # if placemark found
            # select text to be replaced with new todos, beginning just after placemark, before timestamp
            editor.SetSelection(st + len(self.placemark), editor.GetLength())

            sel = editor.GetSelectedText()[0:21]   # get the timestamp from just after the placemark
            result = tscheck.search(sel)
            if result:
                lastCreation = mktime(strptime(sel[4:20], "%Y-%m-%d %H:%M"))   #todo: easier!?!?
            else:
                lastCreation = 0.0

            editor.ReplaceSelection(" (@ %s) %s" % (myNow, self.underline))    # add the new timestamp

            if srchstr != "" and self.filterTodos:                       # optionally add filter info
                editor.AddText(" (filter: '%s')" % srchstr)
            editor.AddText("\n\n\n")

        else:                                           # if no placemark found
            editor.GotoPos(editor.GetLength())          #    add the placemark and todos at end of page
            lastCreation = 0.0
            editor.AddText("\n\n%s (@ %s) %s" % (self.placemark, myNow, self.underline))
            if srchstr != "" and self.filterTodos:
                editor.AddText(" (filter: '%s')" % srchstr)
            editor.AddText("\n\n\n")

        st = editor.FindText(0, editor.GetLength(), "[icon:", 0)
        if st != -1:                                        # if there already is an icon:string:
            self.iconString = ""                                 #   don't add our own

        st = editor.FindText(0, editor.GetLength(), "[color:", 0)
        if st != -1:                                        # if there already is an color:string:
            colorStringFlag = 0                             #   don't add our own


        # determine tags

        checkTags = [tag[0] for tag in self.tags]           # first part

        newtags = []

        for todo in todos:                           ## [(u'pagename', u'todo.tag1.tag2: text')...
            allTodo = todo[1].split(":")             ## [u'todo.tag1.tag2', u' text'] ...

            words = allTodo[0].split(".",1)          ## [u'todo', u'tag1.tag2'] ...
                                                     ## (keep additionally dotted parts together)! o.k.?

            for word in words[1:]:                   ## u'tag1.tag2'
                newtags.append( (word) )

        #---------------------------------------------------

        dateChecks = [
                       ( compile("[0-9]{4}-[0-1][0-9]-[0-3][0-9]$"), "yyyy-mm-dd" ),  # specific date; ISO
                       ( compile("[0-1][0-9]/[0-3][0-9]/[0-9]{4}$"), "mm/dd/yyyy" ),  # specific date; US
                       ( compile("[0-3][0-9].[0-1][0-9].[0-9]{4}$"), "dd.mm.yyyy" ),  # specific date; German
                       ( compile("[0-9]{4}-W[0-9]{2}-[0-7]$"),       "yyyy-Wnn-d" ),  # DayOfWeek "d" is Mon(1)-Sun(7)
                       ( compile("[0-9]{4}-W[0-9]{2}$"),             "yyyy-Wnn" ),    # On Monday of week nn in year yyyy
                       ( compile("\*-W[0-9]{2}$"),                   "*-Wnn" ),       # On Monday of week nn every year 
                       ( compile("CW[0-9]{2}$"),                     "CWnn" ),        #   same as above: Calendar Week.  todo: is "CW" the correct name in English?
                       ( compile("KW[0-9]{2}$"),                     "KWnn" ),        #   same as above: German: Kalender-Woche
                       ( compile("\*-W\*-[0-7]$"),                   "*-W*-d" ),      # every DayOfWeek "d" is Mon(1)-Sun(7)
                       ( compile("\*-[0-1][0-9]-[0-3][0-9]$"),       "*-mm-dd" ),     # every dd'th day of month mm
                       ( compile("\*-\*-[0-3][0-9]$"),               "*-*-dd" ),      # every dd'th day every month
                       ( compile("\*-\*-\*$"),                       "*-*-*" ),       # every day
                     ]

        newtagsdata=[]

        # check all tags for being a date of special format

        for newtag in newtags:

           for dateCheck in dateChecks:            # any match?
               result = dateCheck[0].search(newtag)
               if result: break

           if not result:                          # not a date like the above, perhaps no date at all
               newtagsdata.append( (newtag, newtag, 0, "") )

           else:

               if dateCheck[1] == "yyyy-mm-dd":
                   y  = result.group()[0:4]
                   m  = result.group()[5:7]
                   d  = result.group()[8:10]
                   t  = newtag[0:-10]

               elif dateCheck[1] == "mm/dd/yyyy":
                   m  = result.group()[0:2]
                   d  = result.group()[3:5]
                   y  = result.group()[6:10]
                   t  = newtag[0:-10]

               elif dateCheck[1] == "yyyy-Wnn-d":
                   y  = int(result.group()[0:4])
                   w  = int(result.group()[6:8])
                   dw = int(result.group()[9:10])
                   y,m,d = RevWN(y, w, dw)                   # eval year, month, day from week [and day]
                   t  = newtag[0:-10]

               elif dateCheck[1] == "yyyy-Wnn":
                   y  = int(result.group()[0:4])
                   w  = int(result.group()[6:8])
                   dw = 1
                   y,m,d = RevWN(y, w, dw)
                   t  = newtag[0:-8]

               elif dateCheck[1] == "dd.mm.yyyy":
                   d  = result.group()[0:2]
                   m  = result.group()[3:5]
                   y  = result.group()[6:10]
                   t  = newtag[0:-10]

               elif dateCheck[1] == "KWnn" or dateCheck[1] == "CWnn":
                   w  = int(result.group()[2:4])
                   y,m,d = RevWN(thisyear, w, 1)
                   if w < thisweek or (w == thisweek and thiswday > 1):
                       y = y + 1
                   t  = newtag[0:-4]

               elif dateCheck[1] == "*-Wnn":
                   w  = int(result.group()[3:5])
                   y,m,d = RevWN(thisyear, w, 1)
                   if w < thisweek or (w == thisweek and thiswday > 1):
                       y = y + 1
                   t  = newtag[0:-5]

               elif dateCheck[1] ==  "*-W*-d":
                   dw = int(result.group()[5:6])
                   y,m,d = RevWN(thisyear, thisweek, dw)                   # eval year, month, day from week [and day]
                   if d < thisday:
                       if m < 12  or (m == 12 and d < 25):
                           y,m,d = RevWN(thisyear, thisweek+1, dw)
                       else:
                           y,m,d = RevWN(thisyear+1, 1, dw)
                   t  = newtag[0:-6]   #  + "@@:%d-%d-%d-%d@@" %(thisyear,thisweek,dw,d)

               elif dateCheck[1] == "*-mm-dd":
                   y  = thisyear
                   m  = int(result.group()[2:4])
                   d  = int(result.group()[5:7])
                   if m < thismonth  or (m == thismonth and d < thisday):
                       y = y + 1
                   t  = newtag[0:-7]

               elif dateCheck[1] == "*-*-dd":
                   y  = thisyear
                   m  = thismonth
                   d  = int(result.group()[4:6])
                   if d < thisday:
                       m = m + 1
                       if m > 12:
                           m = 1
                           y = y + 1
                   t  = newtag[0:-6]

               elif dateCheck[1] == "*-*-*":
                   y  = thisyear
                   m  = thismonth
                   d  = thisday
                   t  = newtag[0:-5]

               try:
                   date(int(y), int(m), int(d))
               except ValueError:
                   self.wikidpad.displayMessage("Invalid date!", newtag + "  ==>  y=%s m=%s d=%s" %(y,m,d) + "\n\nPlease fix it manually and try again. Date is set to 'TODAY' now.\n(Later there may be an automatic solution...)")
                   y = thisyear
                   m = thismonth
                   d = thisday

               # with flag = 1 to mark as date entry
               newtagsdata.append( (str(y) + "-" + str(m).zfill(2) + "-" + str(d).zfill(2), newtag, 1, t[0:t.rfind(".")]) )    # todo: save date format for later display


        # ==> newtagsdata = [ ( "yyyy-mm-dd" , "some date string" , 1, "tag(s) before date")  # for date tags
        #                     ( tagstring    , tagstring          , 0, "")                    # for normal tags

        #---------------------------------------------------

        # search for tags not in defined list and automatically add them in correct order

        #if realDatesSeparate:    # perhaps not necessary, but may reduce calculations below?
        #    if isCalendar:
        #        newtagsdata = [ t for t in sorted(newtagsdata) if t[2]==1 ]
        #    else:
        #        newtagsdata = [ t for t in sorted(newtagsdata) if t[2]==0 ]
        #else:
        if self.realDatesLast:
            newtagsdataX1 = [ t for t in sorted(newtagsdata) if t[2]==0 ]
            newtagsdataX2 = [ t for t in sorted(newtagsdata) if t[2]==1 ]

            newtagsdata = newtagsdataX1[:]
            newtagsdata.extend(newtagsdataX2)
        else:
            newtagsdata.sort()


        for sortednewtag in newtagsdata:

            if not sortednewtag[1] in checkTags:     # only if not already in tag list

                if sortednewtag[2] == 0:             # if not a recognized date

                    self.tags.append( (sortednewtag[1], self.notagHeading, sortednewtag[1], "") )

                else:                                # if a date

                    # "2007-10-19" ==> (2007, 42, 5) ==> "Friday Week42"
                    weekDay    = ""
                    weekNumber = ""

                    try:
                        xDate = date(int(sortednewtag[0][0:4]), int(sortednewtag[0][5:7]), int(sortednewtag[0][8:10]))
                        xyear,xweek,xwday = date.isocalendar(xDate)
                        if self.showWeekday:     weekDay    = self.weekdayFormat % self.weekdays[xwday-1]    # because ISO-day begins with 1
                        if self.showWeeknumber:  weekNumber = self.weeknumberFormat % xweek
                    except:
                        weekDay    = " (!!invalid date!!)"
                    # todo: check dates earlier to be sure they are valid!

                    outDate = strftime(self.outDateFormat,strptime(sortednewtag[0], "%Y-%m-%d"))

                    if sortednewtag[0] <= warndate:
                        if sortednewtag[0] < myToday:
                            missedTodos += 1
                            self.tags.append( (sortednewtag[1], self.dateHeadingX, outDate + weekDay + weekNumber + self.missedString, sortednewtag[3] ) )
                        elif sortednewtag[0] == myToday:
                            todosForToday += 1
                            self.tags.append( (sortednewtag[1], self.dateHeadingX, outDate + weekDay + weekNumber + self.nowString, sortednewtag[3] ) )
                        else:
                            todosForNextDays += 1
                            n = (mktime(strptime(sortednewtag[0],"%Y-%m-%d")) - time()) / (60*60*24)
                            plural = ""
                            if n <= 1:
                                if int(n*24) > 1: plural = "s"
                                self.tags.append( (sortednewtag[1], self.dateHeadingX, outDate + weekDay + weekNumber + " ___ " + str(int(n*24)) + " hour" + plural + " left", sortednewtag[3]) )
                            else:
                                if int(n) > 1: plural = "s"
                                self.tags.append( (sortednewtag[1], self.dateHeadingX, outDate + weekDay + weekNumber + " ___ " + str(int(n)) + " day" + plural + " left", sortednewtag[3]) )
                    else:

                        self.tags.append( (sortednewtag[1], self.dateHeading, outDate + weekDay + weekNumber, sortednewtag[3] ) )

                checkTags.append(sortednewtag[1])

        # ==> tags = [ ( "some date string", "heading" , "YYYY-MM-DD ....", "tags before date-string")   # for new dates
        # ==> tags = [ ( "some string",      "heading" , "some string"    , "")                          # for other ones


        #---------------------------------------------------

        # iterate thru all tags now and insert them on page

        lasttag  = u""
        newtext  = u""
        tagAdded = False

        #dcheck = compile("[0-9]{4}-[0-1][0-9]-[0-3][0-9]")   # YYYY-MM-DD
        # trying to honor self.outDateFormat (e.g. "%Y-%m-%d"):   # todo: more flexible
        dexpr = self.outDateFormat.replace("%d","[0-3][0-9]").replace("%m","[0-1][0-9]").replace("%Y","[0-9]{4}").replace("%y","[0-9]{2}")
        dcheck = compile(dexpr)

        if not isCalendar or self.realDatesSeparate:

            for tag in self.tags:

                if self.realDatesSeparate:
                    result = dcheck.search(tag[2])
                    if not isCalendar and result:
                        continue
                    if isCalendar and not result:
                        continue

                tagContinued = (lasttag != "") and (lasttag == tag[2][0:10])

                lasttag = tag[2][0:10]


                # write SPACER to the tag list:

                if tagAdded and not tagContinued:
                    newtext += '\n' + self.spacerline
                    tagAdded = False


                wroteHeader = False   # write header only once


                # handle untagged todos

                if tag[0] == 'UNTAGGED':

                    for todo in sorted(todos):
                        foundTag = False
                        for t in self.tags:          # avoid doubles
                            if t[0] in todo[1]:      # perhaps still not accurate enough??? (see below on tagged todos)
                                foundTag = True
                                break

                        if not foundTag:
                            if not wroteHeader:
                                newtext += '\n' + tag[1] + '\n'
                                wroteHeader = True

                            s = todo[1]
                            if self.deleteAttributes:
                                s = self.delAttribs(s)

                            # handle "unreal" todos (like 'done' ...)
                            s1 = ""
                            if (not self.onlyRealTodos) and (s[0:4] != "todo"):
                                s1 = "  {%s}" % s[0:s.find(':')]

                            # create links
                            ww1 = todo[0]
                            #if not self.wikidpad.getFormatting().isCcWikiWord(ww1):
                            if not langHelper.isCcWikiWord(ww1):
                                ww1 = "[%s]" % ww1

                            if self.extendedLinks:  newtext += "%s%s#%s : %s%s\n"  % (self.bullet, ww1, s.replace(" ","# "), s[s.find(':')+1:], s1)
                            else:                   newtext += "%s%s: %s%s\n"      % (self.bullet, ww1, s[s.find(':')+1:], s1)

                            tagAdded = True

                else:

                # handle tagged todos

                    for todo in sorted(todos):

                        if (tag[0] == todo[1][todo[1].find(".")+1:todo[1].find(":")]):

                            if not wroteHeader and not tagContinued:
                                if   tag[2] == "":     newtext += '\n' + tag[1]            # normal tag of known type
                                elif tag[2] != tag[0]: newtext += '\n' + tag[1] + tag[2]   # tag with date and add. info
                                else:                  newtext += '\n' + tag[1] + tag[0]   # tag of unknown type and std. dates YYYY-MM-DD without info #todo: correct descr.???

                                newtext += '\n'
                                wroteHeader = True

                            s = todo[1]
                            if self.deleteAttributes:
                                s = self.delAttribs(s)

                            # optionally handle "unreal" todos (like 'done' ...)
                            s1 = ""
                            if (not self.onlyRealTodos) and (s[0:4] != "todo"):
                                st,dummy = s.split(':', 1)
                                st,dummy = st.split('.', 1)
                                s1 = "  {" + st + "}"

                            # create the links
                            ww1 = todo[0]
                            #if not self.wikidpad.getFormatting().isCcWikiWord(ww1):
                            if not langHelper.isCcWikiWord(ww1):
                                ww1 = "[%s]" % ww1

                            ti = ""  # possible tags before date string
                            if tag[3] != "":
                              ti = " (%s)" % tag[3]

                            if self.extendedLinks: newtext += "%s%s#%s : %s %s%s\n"  % (self.bullet, ww1, s.replace(" ","# "), ti, s[s.find(':')+1:], s1)
                            else:                  newtext += "%s%s:%s %s%s\n"       % (self.bullet, ww1, ti, s[s.find(':')+1:], s1)

                            tagAdded = True


        # and finally the footer

        newtext += '\n' + self.spacerline

        if self.realDatesSeparate:
            if isCalendar:
                newtext += self.spacerline + "\nDue to configuration option there may be the main todo page: ToDo ..."
            else:
                newtext += self.spacerline + "\nDue to configuration option there may be an additional page: ToDoCalendar ..."
            newtext += '\n' + 2 * self.spacerline

        if self.iconString != "" or colorStringFlag:
            newtext += '\n\n'
            if self.iconString != "":
                newtext += self.iconString + "\n"
            if colorStringFlag:
                if missedTodos:        newtext += self.colorStringMissed
                elif todosForToday:    newtext += self.colorStringToday
                elif todosForNextDays: newtext += self.colorStringNext
                else:                  newtext += self.colorStringNormal

        if self.setBookmark:
            newtext += '\n[bookmarked=true]\n'

        if self.showVersion:
            newtext += "\n*** Created by Todo-Extension version %s in Wikidpad %s ***\n" % (version, wpversion)

        if not self.realDatesSeparate and isCalendar and not tagAdded:
            newtext = "Due to configuration option there are no entries on this page.\nHave a look at the main todo page: ToDo"

        editor.AddText(newtext)    # put all the new text into the editor page

        if self.showInfoPopup:
            if (self.realDatesSeparate and isCalendar) or (not self.realDatesSeparate and not isCalendar):
                if lastCreation < (time() - self.infoPopupTimeDelta * 60):
                    msg = ""
                    if missedTodos:      msg  =   "%d x missed todo.\n"              % missedTodos
                    if todosForToday:    msg += "\n%d x todo for today! \n"          % todosForToday
                    if todosForNextDays: msg += "\n%d x todo for the next %d days!"  % (todosForNextDays, self.dayrange)

                    if msg != "":
                        self.wikidpad.displayMessage("Info from %s" % wikiWord, msg)