From aab32ae9de316bf1451391507369d8e1c978473f Mon Sep 17 00:00:00 2001 From: Matt Mackall Date: Thu, 30 Apr 2009 11:57:52 -0500 Subject: [PATCH 02/58] Add GPLv2+ license and copyright notice --- COPYING | 340 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ smem | 9 ++ 2 files changed, 349 insertions(+) create mode 100644 COPYING diff --git a/COPYING b/COPYING new file mode 100644 index 0000000..d60c31a --- /dev/null +++ b/COPYING @@ -0,0 +1,340 @@ + GNU GENERAL PUBLIC LICENSE + Version 2, June 1991 + + Copyright (C) 1989, 1991 Free Software Foundation, Inc. + 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The licenses for most software are designed to take away your +freedom to share and change it. By contrast, the GNU General Public +License is intended to guarantee your freedom to share and change free +software--to make sure the software is free for all its users. This +General Public License applies to most of the Free Software +Foundation's software and to any other program whose authors commit to +using it. (Some other Free Software Foundation software is covered by +the GNU Library General Public License instead.) You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +this service if you wish), that you receive source code or can get it +if you want it, that you can change the software or use pieces of it +in new free programs; and that you know you can do these things. + + To protect your rights, we need to make restrictions that forbid +anyone to deny you these rights or to ask you to surrender the rights. +These restrictions translate to certain responsibilities for you if you +distribute copies of the software, or if you modify it. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must give the recipients all the rights that +you have. You must make sure that they, too, receive or can get the +source code. And you must show them these terms so they know their +rights. + + We protect your rights with two steps: (1) copyright the software, and +(2) offer you this license which gives you legal permission to copy, +distribute and/or modify the software. + + Also, for each author's protection and ours, we want to make certain +that everyone understands that there is no warranty for this free +software. If the software is modified by someone else and passed on, we +want its recipients to know that what they have is not the original, so +that any problems introduced by others will not reflect on the original +authors' reputations. + + Finally, any free program is threatened constantly by software +patents. We wish to avoid the danger that redistributors of a free +program will individually obtain patent licenses, in effect making the +program proprietary. To prevent this, we have made it clear that any +patent must be licensed for everyone's free use or not licensed at all. + + The precise terms and conditions for copying, distribution and +modification follow. + + GNU GENERAL PUBLIC LICENSE + TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION + + 0. This License applies to any program or other work which contains +a notice placed by the copyright holder saying it may be distributed +under the terms of this General Public License. The "Program", below, +refers to any such program or work, and a "work based on the Program" +means either the Program or any derivative work under copyright law: +that is to say, a work containing the Program or a portion of it, +either verbatim or with modifications and/or translated into another +language. (Hereinafter, translation is included without limitation in +the term "modification".) Each licensee is addressed as "you". + +Activities other than copying, distribution and modification are not +covered by this License; they are outside its scope. The act of +running the Program is not restricted, and the output from the Program +is covered only if its contents constitute a work based on the +Program (independent of having been made by running the Program). +Whether that is true depends on what the Program does. + + 1. You may copy and distribute verbatim copies of the Program's +source code as you receive it, in any medium, provided that you +conspicuously and appropriately publish on each copy an appropriate +copyright notice and disclaimer of warranty; keep intact all the +notices that refer to this License and to the absence of any warranty; +and give any other recipients of the Program a copy of this License +along with the Program. + +You may charge a fee for the physical act of transferring a copy, and +you may at your option offer warranty protection in exchange for a fee. + + 2. You may modify your copy or copies of the Program or any portion +of it, thus forming a work based on the Program, and copy and +distribute such modifications or work under the terms of Section 1 +above, provided that you also meet all of these conditions: + + a) You must cause the modified files to carry prominent notices + stating that you changed the files and the date of any change. + + b) You must cause any work that you distribute or publish, that in + whole or in part contains or is derived from the Program or any + part thereof, to be licensed as a whole at no charge to all third + parties under the terms of this License. + + c) If the modified program normally reads commands interactively + when run, you must cause it, when started running for such + interactive use in the most ordinary way, to print or display an + announcement including an appropriate copyright notice and a + notice that there is no warranty (or else, saying that you provide + a warranty) and that users may redistribute the program under + these conditions, and telling the user how to view a copy of this + License. (Exception: if the Program itself is interactive but + does not normally print such an announcement, your work based on + the Program is not required to print an announcement.) + +These requirements apply to the modified work as a whole. If +identifiable sections of that work are not derived from the Program, +and can be reasonably considered independent and separate works in +themselves, then this License, and its terms, do not apply to those +sections when you distribute them as separate works. But when you +distribute the same sections as part of a whole which is a work based +on the Program, the distribution of the whole must be on the terms of +this License, whose permissions for other licensees extend to the +entire whole, and thus to each and every part regardless of who wrote it. + +Thus, it is not the intent of this section to claim rights or contest +your rights to work written entirely by you; rather, the intent is to +exercise the right to control the distribution of derivative or +collective works based on the Program. + +In addition, mere aggregation of another work not based on the Program +with the Program (or with a work based on the Program) on a volume of +a storage or distribution medium does not bring the other work under +the scope of this License. + + 3. You may copy and distribute the Program (or a work based on it, +under Section 2) in object code or executable form under the terms of +Sections 1 and 2 above provided that you also do one of the following: + + a) Accompany it with the complete corresponding machine-readable + source code, which must be distributed under the terms of Sections + 1 and 2 above on a medium customarily used for software interchange; or, + + b) Accompany it with a written offer, valid for at least three + years, to give any third party, for a charge no more than your + cost of physically performing source distribution, a complete + machine-readable copy of the corresponding source code, to be + distributed under the terms of Sections 1 and 2 above on a medium + customarily used for software interchange; or, + + c) Accompany it with the information you received as to the offer + to distribute corresponding source code. (This alternative is + allowed only for noncommercial distribution and only if you + received the program in object code or executable form with such + an offer, in accord with Subsection b above.) + +The source code for a work means the preferred form of the work for +making modifications to it. For an executable work, complete source +code means all the source code for all modules it contains, plus any +associated interface definition files, plus the scripts used to +control compilation and installation of the executable. However, as a +special exception, the source code distributed need not include +anything that is normally distributed (in either source or binary +form) with the major components (compiler, kernel, and so on) of the +operating system on which the executable runs, unless that component +itself accompanies the executable. + +If distribution of executable or object code is made by offering +access to copy from a designated place, then offering equivalent +access to copy the source code from the same place counts as +distribution of the source code, even though third parties are not +compelled to copy the source along with the object code. + + 4. You may not copy, modify, sublicense, or distribute the Program +except as expressly provided under this License. Any attempt +otherwise to copy, modify, sublicense or distribute the Program is +void, and will automatically terminate your rights under this License. +However, parties who have received copies, or rights, from you under +this License will not have their licenses terminated so long as such +parties remain in full compliance. + + 5. You are not required to accept this License, since you have not +signed it. However, nothing else grants you permission to modify or +distribute the Program or its derivative works. These actions are +prohibited by law if you do not accept this License. Therefore, by +modifying or distributing the Program (or any work based on the +Program), you indicate your acceptance of this License to do so, and +all its terms and conditions for copying, distributing or modifying +the Program or works based on it. + + 6. Each time you redistribute the Program (or any work based on the +Program), the recipient automatically receives a license from the +original licensor to copy, distribute or modify the Program subject to +these terms and conditions. You may not impose any further +restrictions on the recipients' exercise of the rights granted herein. +You are not responsible for enforcing compliance by third parties to +this License. + + 7. If, as a consequence of a court judgment or allegation of patent +infringement or for any other reason (not limited to patent issues), +conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot +distribute so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you +may not distribute the Program at all. For example, if a patent +license would not permit royalty-free redistribution of the Program by +all those who receive copies directly or indirectly through you, then +the only way you could satisfy both it and this License would be to +refrain entirely from distribution of the Program. + +If any portion of this section is held invalid or unenforceable under +any particular circumstance, the balance of the section is intended to +apply and the section as a whole is intended to apply in other +circumstances. + +It is not the purpose of this section to induce you to infringe any +patents or other property right claims or to contest validity of any +such claims; this section has the sole purpose of protecting the +integrity of the free software distribution system, which is +implemented by public license practices. Many people have made +generous contributions to the wide range of software distributed +through that system in reliance on consistent application of that +system; it is up to the author/donor to decide if he or she is willing +to distribute software through any other system and a licensee cannot +impose that choice. + +This section is intended to make thoroughly clear what is believed to +be a consequence of the rest of this License. + + 8. If the distribution and/or use of the Program is restricted in +certain countries either by patents or by copyrighted interfaces, the +original copyright holder who places the Program under this License +may add an explicit geographical distribution limitation excluding +those countries, so that distribution is permitted only in or among +countries not thus excluded. In such case, this License incorporates +the limitation as if written in the body of this License. + + 9. The Free Software Foundation may publish revised and/or new versions +of the General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + +Each version is given a distinguishing version number. If the Program +specifies a version number of this License which applies to it and "any +later version", you have the option of following the terms and conditions +either of that version or of any later version published by the Free +Software Foundation. If the Program does not specify a version number of +this License, you may choose any version ever published by the Free Software +Foundation. + + 10. If you wish to incorporate parts of the Program into other free +programs whose distribution conditions are different, write to the author +to ask for permission. For software which is copyrighted by the Free +Software Foundation, write to the Free Software Foundation; we sometimes +make exceptions for this. Our decision will be guided by the two goals +of preserving the free status of all derivatives of our free software and +of promoting the sharing and reuse of software generally. + + NO WARRANTY + + 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY +FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN +OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES +PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED +OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS +TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE +PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, +REPAIR OR CORRECTION. + + 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR +REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, +INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING +OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED +TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY +YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER +PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE +POSSIBILITY OF SUCH DAMAGES. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +convey the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 2 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + + +Also add information on how to contact you by electronic and paper mail. + +If the program is interactive, make it output a short notice like this +when it starts in an interactive mode: + + Gnomovision version 69, Copyright (C) year name of author + Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, the commands you use may +be called something other than `show w' and `show c'; they could even be +mouse-clicks or menu items--whatever suits your program. + +You should also get your employer (if you work as a programmer) or your +school, if any, to sign a "copyright disclaimer" for the program, if +necessary. Here is a sample; alter the names: + + Yoyodyne, Inc., hereby disclaims all copyright interest in the program + `Gnomovision' (which makes passes at compilers) written by James Hacker. + + , 1 April 1989 + Ty Coon, President of Vice + +This General Public License does not permit incorporating your program into +proprietary programs. If your program is a subroutine library, you may +consider it more useful to permit linking proprietary applications with the +library. If this is what you want to do, use the GNU Library General +Public License instead of this License. diff --git a/smem b/smem index c4cffd8..6d91a5d 100755 --- a/smem +++ b/smem @@ -1,4 +1,13 @@ #!/usr/bin/python +# +# smem - a tool for meaningful memory reporting +# +# Copyright 2008-2009 Matt Mackall +# +# This software may be used and distributed according to the terms of +# the GNU General Public License version 2 or later, incorporated +# herein by reference. + import re, os, sys, pwd, grp, optparse, errno, tarfile _ucache = {} From 52848c0efbff541c116fcdf0af56415574c24523 Mon Sep 17 00:00:00 2001 From: Jeff Schroeder Date: Thu, 30 Apr 2009 20:04:20 -0500 Subject: [PATCH 03/58] Kernel version >= 2.6.27 check Awesome tool! I learned about this from the LWN article and immediately (stupidly) tried it out on a centos 5 host. Here is a patch to add a kernel version check. --- smem | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/smem b/smem index 6d91a5d..f15891c 100755 --- a/smem +++ b/smem @@ -548,6 +548,13 @@ def showbar(l, columns, sort): pylab.legend([p[0] for p in pl], key) pylab.show() +def kernel_version_check(): + kernel_release = os.uname()[2].split('-')[0] + if kernel_release < "2.6.27": + name = os.path.basename(sys.argv[0]) + sys.stderr.write(name + " requires a kernel >= 2.6.27\n") + sys.exit(-1) + parser = optparse.OptionParser("%prog [options]") parser.add_option("-H", "--no-header", action="store_true", help="disable header line") @@ -596,6 +603,7 @@ parser.add_option("-S", "--source", type="str", help="/proc data source") +kernel_version_check() defaults = {} parser.set_defaults(**defaults) (options, args) = parser.parse_args() From 4865bf296757d0041c98d713abf61cbf497fca95 Mon Sep 17 00:00:00 2001 From: ??? Date: Thu, 14 May 2009 22:08:38 -0500 Subject: [PATCH 04/58] [PATCH] physical memory size computing error There are two minor bugs on physical memory size computing: (1) fromunits() returns wrong value for "2001844kB", which consists of more than one digits. (2) memory()['memtotal'] is in kB. If "--realmem" is not provided at CLI, totalmem() returns number in MB and the "firmware/hardware" amount will be minus. totalmem() needs to always return value in kB. Here is how to test this patch: (1) Do "smem -w" , the "firmware/hardware" amount should not be minus. (2) Do "smem -w -R 2001844kB" (change 2001844kB per your PC, note to keep it in kB unit) , the "firmware/hardware" amount should not be minus. Here's a patch to fix these issues. --- smem | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/smem b/smem index f15891c..4d4f2e1 100755 --- a/smem +++ b/smem @@ -77,7 +77,7 @@ def totalmem(): _totalmem = fromunits(options.realmem) else: _totalmem = memory()['memtotal'] - return _totalmem / 1024 + return _totalmem _kernelsize = 0 def kernelsize(): From 04b9f552f420c82564774260518b06aa7bd6d64e Mon Sep 17 00:00:00 2001 From: ??? Date: Thu, 14 May 2009 22:22:44 -0500 Subject: [PATCH 05/58] [PATCH] invalid "-K" value cause smem ended with IndexError exception I do "smem -w -K a.txt -R 2048M" and got following error: zhichyu@w-shpd-zcyu:~/sftw4ubuntu$ smem-0.1/smem -w -K a.txt -R 2048M size: 'a.txt': No such file Traceback (most recent call last): File "smem-0.1/smem", line 607, in showsystem() File "smem-0.1/smem", line 361, in showsystem k = kernelsize() File "smem-0.1/smem", line 77, in kernelsize d = os.popen("size %s" % options.kernel).readlines()[1] IndexError: list index out of range The root cause is that os.popen("size a.txt") returns only one line. If the user provides an invalid kernel image file path, I think it's better to assume the image size is zero than raise an exception. --- smem | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/smem b/smem index 4d4f2e1..fa3275b 100755 --- a/smem +++ b/smem @@ -83,8 +83,11 @@ _kernelsize = 0 def kernelsize(): global _kernelsize if not _kernelsize and options.kernel: - d = os.popen("size %s" % options.kernel).readlines()[1] - _kernelsize = int(d.split()[3]) / 1024 + try: + d = os.popen("size %s" % options.kernel).readlines()[1] + _kernelsize = int(d.split()[3]) / 1024 + except: + pass return _kernelsize def pidmaps(pid): From b2041ff9b6ca2c2af3935ed169b52f3bdb0b07d6 Mon Sep 17 00:00:00 2001 From: Ademar de Souza Reis Jr Date: Thu, 21 May 2009 11:46:37 -0300 Subject: [PATCH 06/58] Fix broken -n option [ademar@optimus smem]$ ./smem -n Traceback (most recent call last): File "./smem", line 624, in ... --- smem | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/smem b/smem index fa3275b..d4a0187 100755 --- a/smem +++ b/smem @@ -216,7 +216,7 @@ def showpids(): def showuser(p): if options.numeric: - return piduser(p) + return src.piduser(p) return pidusername(p) fields = dict( From 67aee6e39d2da3dda6a035c4fe447cd1dabcf349 Mon Sep 17 00:00:00 2001 From: Tim Bird Date: Fri, 22 May 2009 12:41:07 -0500 Subject: [PATCH 07/58] Kernel version >= 2.6.27 check Jeff Schroeder wrote: > Awesome tool! I learned about this from the LWN article and > immediately (stupidly) tried it out on a centos 5 host. Here is a > patch to add a kernel version check. This is a nice fix, but the version check should be done against the proc data being used (which is not necessarily that of the local kernel). This required moving kernel_version_check to after where the src data is read. --- capture | 2 +- smem | 7 +++++-- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/capture b/capture index e23995c..e0d32d4 100755 --- a/capture +++ b/capture @@ -4,7 +4,7 @@ # capture a memory data snapshot with realtime priority mkdir -p $1 chrt --fifo 99 \ - cp -a --parents /proc/[0-9]*/{smaps,cmdline,stat} /proc/meminfo $1 + cp -a --parents /proc/[0-9]*/{smaps,cmdline,stat} /proc/meminfo /proc/version $1 # build a compressed tarball of snapshot cd $1/proc diff --git a/smem b/smem index d4a0187..4caac37 100755 --- a/smem +++ b/smem @@ -33,6 +33,8 @@ class procdata(object): return self._readlines('%s/smaps' % pid) def memdata(self): return self._readlines('meminfo') + def version(self): + return self._readlines('version')[0] def pidname(self, pid): l = self._read('%d/stat' % pid) return l[l.find('(') + 1: l.find(')')] @@ -552,7 +554,7 @@ def showbar(l, columns, sort): pylab.show() def kernel_version_check(): - kernel_release = os.uname()[2].split('-')[0] + kernel_release = src.version().split()[2].split('-')[0] if kernel_release < "2.6.27": name = os.path.basename(sys.argv[0]) sys.stderr.write(name + " requires a kernel >= 2.6.27\n") @@ -606,7 +608,6 @@ parser.add_option("-S", "--source", type="str", help="/proc data source") -kernel_version_check() defaults = {} parser.set_defaults(**defaults) (options, args) = parser.parse_args() @@ -616,6 +617,8 @@ try: except: src = procdata(options.source) +kernel_version_check() + try: if options.mappings: showmaps() From dac4809c064ea9fe0b51f315f3d06779437dd444 Mon Sep 17 00:00:00 2001 From: Matt Mackall Date: Fri, 22 May 2009 17:29:51 -0500 Subject: [PATCH 08/58] be less picky about tar directories --- smem | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/smem b/smem index 4caac37..39debf3 100755 --- a/smem +++ b/smem @@ -60,8 +60,9 @@ class tardata(procdata): self.tar = tarfile.open(source) def _list(self): for ti in self.tar: - if ti.name.endswith('/'): - yield ti.name[:-1] + if ti.name.endswith('/smaps'): + d,f = ti.name.split('/') + yield d def _read(self, f): return self.tar.extractfile(f).read() def _readlines(self, f): From 383d6480dff925cefa3f587275685deb8b781145 Mon Sep 17 00:00:00 2001 From: Matt Mackall Date: Fri, 22 May 2009 17:31:54 -0500 Subject: [PATCH 09/58] add smemcap tool --- smemcap.c | 112 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 112 insertions(+) create mode 100644 smemcap.c diff --git a/smemcap.c b/smemcap.c new file mode 100644 index 0000000..bcdfc29 --- /dev/null +++ b/smemcap.c @@ -0,0 +1,112 @@ +/* + smem - a tool for meaningful memory reporting + + Copyright 2008-2009 Matt Mackall + + This software may be used and distributed according to the terms of + the GNU General Public License version 2 or later, incorporated + herein by reference. +*/ + +#include +#include +#include +#include +#include +#include +#include +#include + +struct fileblock; +struct fileblock { + char data[512]; + struct fileblock *next; +}; + +int writeheader(int destfd, const char *path, int mode, int uid, int gid, + int size, int mtime, int type) +{ + char header[512]; + int i, sum; + + memset(header, 0, 512); + + sprintf(header, "%s", path); + sprintf(header + 100, "%08o", mode); + sprintf(header + 108, "%08o", uid); + sprintf(header + 116, "%08o", gid); + sprintf(header + 124, "%012o", size); + sprintf(header + 136, "%08o", mtime); + sprintf(header + 148, " %1d", type); + + /* fix checksum */ + for (i = sum = 0; i < 512; i++) + sum += header[i]; + sprintf(header + 148, "%06o\0 %1d", sum, type); + + return write(destfd, header, 512); +} + +int archivefile(const char *path, int destfd) +{ + struct fileblock *start, *cur; + struct fileblock **prev = &start; + int fd, r, size = 0; + struct stat s; + + /* buffer and stat the file */ + fd = open(path, O_RDONLY); + fstat(fd, &s); + + do { + cur = calloc(1, sizeof(struct fileblock)); + *prev = cur; + prev = &cur->next; + r = read(fd, cur->data, 512); + if (r > 0) + size += r; + } while (r == 512); + + close(fd); + + /* write archive header */ + writeheader(destfd, path, s.st_mode, s.st_uid, s.st_gid, size, + s.st_mtime, '0'); + + /* dump file contents */ + for (cur = start; size > 0; size -= 512) { + write(destfd, cur->data, 512); + start = cur; + cur = cur->next; + free(start); + } +} + +int archivejoin(const char *sub, const char *name, int destfd) +{ + char path[256]; + sprintf(path, "%s/%s", sub, name); + return archivefile(path, destfd); +} + +int main(int argc, char *argv[]) +{ + int fd; + DIR *d; + struct dirent *de; + + chdir("/proc"); + archivefile("meminfo", 1); + archivefile("version", 1); + + d = opendir("."); + while (de = readdir(d)) + if (de->d_name[0] >= '0' && de->d_name[0] <= '9') { + writeheader(1, de->d_name, 040555, 0, 0, 0, 0, 5); + archivejoin(de->d_name, "smaps", 1); + archivejoin(de->d_name, "cmdline", 1); + archivejoin(de->d_name, "stat", 1); + } + + return 0; +} From c8bb8425b226c21e13eebecae3795898abf498ca Mon Sep 17 00:00:00 2001 From: Matt Mackall Date: Wed, 27 May 2009 18:12:00 -0500 Subject: [PATCH 10/58] Make system memory reporting more robust - totalmem should return kB when provided manually - firmware size never goes below zero - add comments - calculate kernel portion of cached by subtracting mapped rather than anonymous - get rid of sum() bits for silly column totals --- smem | 26 +++++++++++++++++--------- 1 file changed, 17 insertions(+), 9 deletions(-) diff --git a/smem b/smem index 39debf3..efe836b 100755 --- a/smem +++ b/smem @@ -77,7 +77,7 @@ def totalmem(): global _totalmem if not _totalmem: if options.realmem: - _totalmem = fromunits(options.realmem) + _totalmem = fromunits(options.realmem) / 1024 else: _totalmem = memory()['memtotal'] return _totalmem @@ -373,26 +373,34 @@ def showusers(): def showsystem(): t = totalmem() - k = kernelsize() + ki = kernelsize() m = memory() + mt = m['memtotal'] - fh = t - mt - k f = m['memfree'] + + # total amount used by hardware + fh = max(t - mt - ki, 0) + + # total amount mapped into userspace (ie mapped an unmapped pages) u = m['anonpages'] + m['mapped'] + + # total amount allocated by kernel not for userspace kd = mt - f - u - kdd = (m['buffers'] + m['sreclaimable'] + - (m['cached'] - m['anonpages'])) + + # total amount in kernel caches + kdc = m['buffers'] + m['sreclaimable'] + (m['cached'] - m['mapped']) l = [("firmware/hardware", fh, 0), - ("kernel image", k, 0), - ("kernel dynamic memory", kd, kdd), + ("kernel image", ki, 0), + ("kernel dynamic memory", kd, kdc), ("userspace memory", u, m['mapped']), ("free memory", f, f)] fields = dict( - order=('Order', lambda n: n, '% 1s', lambda x: len(p), + order=('Order', lambda n: n, '% 1s', lambda x: '', 'hierarchical order'), - area=('Area', lambda n: l[n][0], '%-24s', lambda x: len(l), + area=('Area', lambda n: l[n][0], '%-24s', lambda x: '', 'memory area'), used=('Used', lambda n: l[n][1], '%10a', sum, 'area in use'), From a0447bc26b89688009289af540c4a21ccc8d8702 Mon Sep 17 00:00:00 2001 From: Matt Mackall Date: Mon, 8 Jun 2009 15:15:37 -0500 Subject: [PATCH 11/58] Fix some tar header issues for smemcap --- smemcap.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/smemcap.c b/smemcap.c index bcdfc29..d8af86f 100644 --- a/smemcap.c +++ b/smemcap.c @@ -32,11 +32,11 @@ int writeheader(int destfd, const char *path, int mode, int uid, int gid, memset(header, 0, 512); sprintf(header, "%s", path); - sprintf(header + 100, "%08o", mode); - sprintf(header + 108, "%08o", uid); - sprintf(header + 116, "%08o", gid); - sprintf(header + 124, "%012o", size); - sprintf(header + 136, "%08o", mtime); + sprintf(header + 100, "%07o\0", mode & 0777); + sprintf(header + 108, "%07o\0", uid); + sprintf(header + 116, "%07o\0", gid); + sprintf(header + 124, "%011o\0", size); + sprintf(header + 136, "%07o\0", mtime); sprintf(header + 148, " %1d", type); /* fix checksum */ @@ -70,8 +70,8 @@ int archivefile(const char *path, int destfd) close(fd); /* write archive header */ - writeheader(destfd, path, s.st_mode, s.st_uid, s.st_gid, size, - s.st_mtime, '0'); + writeheader(destfd, path, s.st_mode, s.st_uid, + s.st_gid, size, s.st_mtime, 0); /* dump file contents */ for (cur = start; size > 0; size -= 512) { @@ -102,7 +102,7 @@ int main(int argc, char *argv[]) d = opendir("."); while (de = readdir(d)) if (de->d_name[0] >= '0' && de->d_name[0] <= '9') { - writeheader(1, de->d_name, 040555, 0, 0, 0, 0, 5); + writeheader(1, de->d_name, 0555, 0, 0, 0, 0, 5); archivejoin(de->d_name, "smaps", 1); archivejoin(de->d_name, "cmdline", 1); archivejoin(de->d_name, "stat", 1); From 7650aa076ddb9297e07fdbfdabc97d370b614cb1 Mon Sep 17 00:00:00 2001 From: Matt Mackall Date: Tue, 23 Jun 2009 17:01:12 -0500 Subject: [PATCH 12/58] tar source: use usernames and groupnames from tarfile if available --- smem | 37 +++++++++++++++++-------------------- 1 file changed, 17 insertions(+), 20 deletions(-) diff --git a/smem b/smem index efe836b..1168fac 100755 --- a/smem +++ b/smem @@ -10,11 +10,10 @@ import re, os, sys, pwd, grp, optparse, errno, tarfile -_ucache = {} -_gcache = {} - class procdata(object): def __init__(self, source): + self._ucache = {} + self._gcache = {} self.source = source and source or "" def _list(self): return os.listdir(self.source + "/proc") @@ -56,7 +55,7 @@ class procdata(object): class tardata(procdata): def __init__(self, source): - self.source = source + procdata.__init__(self, source) self.tar = tarfile.open(source) def _list(self): for ti in self.tar: @@ -68,9 +67,19 @@ class tardata(procdata): def _readlines(self, f): return self.tar.extractfile(f).readlines() def piduser(self, p): - return self.tar.getmember("%s/cmdline" % p).uid + t = self.tar.getmember("%s/cmdline" % p) + if t.uname: + self._ucache[t.uid] = t.uname + return t.uid def pidgroup(self, p): - return self.tar.getmember("%s/cmdline" % p).gid + t = self.tar.getmember("%s/cmdline" % p) + if t.gname: + self._gcache[t.gid] = t.gname + return t.gid + def username(self, u): + return self._ucache.get(u, str(u)) + def groupname(self, g): + return self._gcache.get(g, str(g)) _totalmem = 0 def totalmem(): @@ -155,20 +164,8 @@ def fromunits(x): if x.endswith(k): return int(float(x[:len(k)])*v) -_ucache = {} -def username(uid): - if uid not in _ucache: - _ucache[uid] = pwd.getpwuid(uid)[0] - return _ucache[uid] - def pidusername(pid): - return username(src.piduser(pid)) - -_gcache = {} -def groupname(gid): - if gid not in _gcache: - _gcache[gid] = grp.getgrgid(gid)[0] - return _gcache[gid] + return src.username(src.piduser(pid)) def showamount(a): if options.abbreviate: @@ -348,7 +345,7 @@ def showusers(): def showuser(u): if options.numeric: return u - return username(u) + return src.username(u) fields = dict( user=('User', showuser, '%-8s', None, From dbe27ffe29f437e9cc4177ee78c3c4d26d42129e Mon Sep 17 00:00:00 2001 From: Matt Mackall Date: Mon, 6 Jul 2009 15:20:05 -0500 Subject: [PATCH 13/58] Fix references to _ucache and _gcache --- smem | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/smem b/smem index 1168fac..654c375 100755 --- a/smem +++ b/smem @@ -46,12 +46,12 @@ class procdata(object): return self._stat('%d/cmdline' % pid).st_gid def username(self, uid): if uid not in _ucache: - _ucache[uid] = pwd.getpwuid(uid)[0] - return _ucache[uid] + self._ucache[uid] = pwd.getpwuid(uid)[0] + return self._ucache[uid] def groupname(self, gid): - if gid not in _gcache: - _gcache[gid] = pwd.getgrgid(gid)[0] - return _gcache[gid] + if gid not in self._gcache: + self._gcache[gid] = pwd.getgrgid(gid)[0] + return self._gcache[gid] class tardata(procdata): def __init__(self, source): From d3d782beb003973090cceea9d0e024b850d9dc08 Mon Sep 17 00:00:00 2001 From: Matt Mackall Date: Mon, 6 Jul 2009 15:20:41 -0500 Subject: [PATCH 14/58] Fix _ucache some more --- smem | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/smem b/smem index 654c375..66e7a91 100755 --- a/smem +++ b/smem @@ -45,7 +45,7 @@ class procdata(object): def pidgroup(self, pid): return self._stat('%d/cmdline' % pid).st_gid def username(self, uid): - if uid not in _ucache: + if uid not in self._ucache: self._ucache[uid] = pwd.getpwuid(uid)[0] return self._ucache[uid] def groupname(self, gid): From 6877fc1daaddffecb7bf8fdb1898031ffefbd3c1 Mon Sep 17 00:00:00 2001 From: lethargo Date: Wed, 15 Jul 2009 17:16:34 -0500 Subject: [PATCH 15/58] add smem man page A while back Matthew Miller asked about a man page for smem. Here is a proposed start of one. --- smem.8 | 157 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 157 insertions(+) create mode 100644 smem.8 diff --git a/smem.8 b/smem.8 new file mode 100644 index 0000000..b66c789 --- /dev/null +++ b/smem.8 @@ -0,0 +1,157 @@ +.TH SMEM 8 "07/09/2009" "" "" + +.SH NAME +smem \- Report memory usage with shared memory divided proportionally. + +.SH SYNOPSIS +.B smem +.RI [ options ] + +.SH DESCRIPTION +\fBsmem\fP reports physical memory usage, taking shared memory pages +into account. Unshared memory is reported as the USS (Unique Set Size). +Shared memory is divided evenly among the processes sharing that memory. +The unshared memory (USS) plus a process's proportion of shared memory +is reported as the PSS (Proportional Set Size). The USS and PSS only +include physical memory usage. They do not include memory that has been +swapped out to disk. + +Memory can be reported by process, by user, by mapping, or system\-wide. +Both text mode and graphical output are available. + +.SH OPTIONS + +.SS GENERAL OPTIONS + +.TP +.B \-h, \-\-help +Show help. + +.SS SOURCE DATA +By default, smem will pull most of the data it needs from the /proc +filesystem of the system it is running on. The \-\-source option lets +you used a tarred set of /proc data saved earlier, possibly on a +different machine. The \-\-kernel and \-\-realmem options let you +specify a couple things that smem cannot discover on its own. + +.TP +.BI "\-K " KERNEL ", \-\-kernel=" KERNEL +Path to kernel image. This lets smem include the size of the kernel's +code and statically allocated data in the systemwide (-w) output. + +.TP +.BI "\-R " REALMEM ", \-\-realmem=" REALMEM +Amount of physical RAM. This lets smem detect the amount of memory used +by firmware/hardware in the systemwide (-w) output. If provided, it +will also be used as the total memory size to base percentages on. + +.TP +.BI "\-S " SOURCE ", \-\-source=" SOURCE +/proc data source. This lets you specify an alternate source of the +/proc data. For example, you can capture data from an embedded system +using smemcap, and parse the data later on a different machine. If the +\-\-source option is not included, smem reports memory usage on the +running system. + +.SS REPORT BY +If none of the following options are include, smem reports memory usage +by process. + +.TP +.B \-m, \-\-mappings +Report memory usage by mapping. + +.TP +.B \-u, \-\-users +Report memory usage by user. + +.TP +.B \-w, \-\-system +Report systemwide memory usage summary. + +.SS FILTER BY +If none of these options are included, memory usage is reported for all +processes, users, or mappings. (Note: If you are running as a non-root +user, and if you are not using the \-\-source options, then you will +only see data from processes whose /proc/ information you have access +to.) + +.TP +.BI "\-M " MAPFILTER ", \-\-mapfilter=" MAPFILTER +Mapping filter regular expression. + +.TP +.BI "\-P " PROCESSFILTER ", \-\-processfilter=" PROCESSFILTER +Process filter regular expression. + +.TP +.BI "\-U " USERFILTER ", \-\-userfilter=" USERFILTER +User filter regular expression. + +.SS OUTPUT FORMATTING + +.TP +.BI "\-c " COLUMNS ", \-\-columns=" COLUMNS +Columns to show. + +.TP +.B \-H, \-\-no\-header +Disable header line. + +.TP +.B \-k, \-\-abbreviate +Show unit suffixes. + +.TP +.B \-n, \-\-numeric +Show numeric user IDs instead of usernames. + +.TP +.B \-p, \-\-percent +Show percentages. + +.TP +.B \-r, \-\-reverse +Reverse sort. + +.TP +.BI "\-s " SORT ", \-\-sort=" SORT +Field to sort on. + +.TP +.B \-t, \-\-totals +Show totals. + +.SS OUTPUT TYPE +These options specify graphical output styles. + +.TP +.BI "\-\-bar=" BAR +Show bar graph. + +.TP +.BI "\-\-pie=" PIE +Show pie graph. + +.PP + +.SH NOTES +\fBsmem\fP requires a 2.6.27 or newer kernel. + +.SH FILES +.I /proc/$pid/cmdline +.PP +.I /proc/$pid/smaps +.PP +.I /proc/$pid/stat +.PP +.I /proc/meminfo +.PP +.I /proc/version + +.SH "SEE ALSO" +.BR free (1), pmap (1) + +.SH AUTHOR +\fBsmem\fP was written by Matt Mackall. + From 5777a0c9c328befbc53973a322590f6b08815be6 Mon Sep 17 00:00:00 2001 From: Matt Mackall Date: Mon, 29 Mar 2010 16:19:21 -0500 Subject: [PATCH 17/58] Drop obsolete capture script --- capture | 12 ------------ 1 file changed, 12 deletions(-) delete mode 100755 capture diff --git a/capture b/capture deleted file mode 100755 index e0d32d4..0000000 --- a/capture +++ /dev/null @@ -1,12 +0,0 @@ -#!/bin/sh -# example of capturing target data for smem - -# capture a memory data snapshot with realtime priority -mkdir -p $1 -chrt --fifo 99 \ - cp -a --parents /proc/[0-9]*/{smaps,cmdline,stat} /proc/meminfo /proc/version $1 - -# build a compressed tarball of snapshot -cd $1/proc -tar czf ../../$1.tgz * - From 9ad7a9f60cec6879c45b2b108d1e2a7f104a04d9 Mon Sep 17 00:00:00 2001 From: Michal ?iha? Date: Sat, 2 Jan 2010 15:53:04 +0100 Subject: [PATCH 18/58] Escape dashes in man page there are two dashes in man page which were forgotten to be escaped. Attached patch fixes it. --- smem.8 | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/smem.8 b/smem.8 index b66c789..ce24bf4 100644 --- a/smem.8 +++ b/smem.8 @@ -37,14 +37,14 @@ specify a couple things that smem cannot discover on its own. .TP .BI "\-K " KERNEL ", \-\-kernel=" KERNEL Path to kernel image. This lets smem include the size of the kernel's -code and statically allocated data in the systemwide (-w) output. +code and statically allocated data in the systemwide (\-w) output. + .TP .BI "\-R " REALMEM ", \-\-realmem=" REALMEM Amount of physical RAM. This lets smem detect the amount of memory used -by firmware/hardware in the systemwide (-w) output. If provided, it +by firmware/hardware in the systemwide (\-w) output. If provided, it will also be used as the total memory size to base percentages on. - .TP .BI "\-S " SOURCE ", \-\-source=" SOURCE /proc data source. This lets you specify an alternate source of the From 4bd765bc0c4d8892d675c9634faae130d3502d03 Mon Sep 17 00:00:00 2001 From: Dean Peterson Date: Mon, 29 Mar 2010 16:38:31 -0500 Subject: [PATCH 19/58] man page patch, including embedded section mentioning smemcap Here is a patch for the smem man page. It includes the following: * A new section on embedded usage briefly describing smemcap NOTE: Someone please doublecheck it, since I am not an embedded developer. * Mentions that kernel image for -K option must be uncompressed. * A new copyright section. * A new resources section. * Replaces notes with a requirements section. * Adds a couple of commands to the see also list. * Fixes a couple typos. --- smem.8 | 69 ++++++++++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 60 insertions(+), 9 deletions(-) diff --git a/smem.8 b/smem.8 index ce24bf4..43418a4 100644 --- a/smem.8 +++ b/smem.8 @@ -1,4 +1,4 @@ -.TH SMEM 8 "07/09/2009" "" "" +.TH SMEM 8 "03/15/2010" "" "" .SH NAME smem \- Report memory usage with shared memory divided proportionally. @@ -16,7 +16,7 @@ is reported as the PSS (Proportional Set Size). The USS and PSS only include physical memory usage. They do not include memory that has been swapped out to disk. -Memory can be reported by process, by user, by mapping, or system\-wide. +Memory can be reported by process, by user, by mapping, or systemwide. Both text mode and graphical output are available. .SH OPTIONS @@ -34,17 +34,21 @@ you used a tarred set of /proc data saved earlier, possibly on a different machine. The \-\-kernel and \-\-realmem options let you specify a couple things that smem cannot discover on its own. + .TP .BI "\-K " KERNEL ", \-\-kernel=" KERNEL -Path to kernel image. This lets smem include the size of the kernel's -code and statically allocated data in the systemwide (\-w) output. - +Path to an uncompressed kernel image. This lets smem include the size +of the kernel's code and statically allocated data in the systemwide +(\-w) output. (To obtain an uncompressed image of a kernel on disk, you +may need to build the kernel yourself, then locate file vmlinux in the +source tree.) .TP .BI "\-R " REALMEM ", \-\-realmem=" REALMEM Amount of physical RAM. This lets smem detect the amount of memory used by firmware/hardware in the systemwide (\-w) output. If provided, it will also be used as the total memory size to base percentages on. + .TP .BI "\-S " SOURCE ", \-\-source=" SOURCE /proc data source. This lets you specify an alternate source of the @@ -54,7 +58,7 @@ using smemcap, and parse the data later on a different machine. If the running system. .SS REPORT BY -If none of the following options are include, smem reports memory usage +If none of the following options are included, smem reports memory usage by process. .TP @@ -135,8 +139,38 @@ Show pie graph. .PP -.SH NOTES -\fBsmem\fP requires a 2.6.27 or newer kernel. +.SH REQUIREMENTS +\fBsmem\fP requires: + +.IP \(bu 3 +Linux kernel 2.6.27 or newer. +.IP \(bu +Python 2.x (at least 2.4 or so). +.IP \(bu +The matplotlib library +(only if you want to generate graphical charts). + +.SH EMBEDDED USAGE +To capture memory statistics on resource\-constrained systems, the +the \fBsmem\fP source includes a utility named \fBsmemcap\fP. +\fBsmemcap\fP captures all /proc entries required by \fBsmem\fP +and outputs them as an uncompressed .tar file to STDOUT. +\fBsmem\fP can analyze the output using the \fB\-\-source\fP option. +\fBsmemcap\fP is small and does not require Python. +.PP +To use \fBsmemcap\fP: +.IP 1. 3 +Obtain the smem source at http://selenic.com/repo/smem +.IP 2. +Compile \fIsmemcap.c\fP for your target system. +.IP 3. +Run \fBsmemcap\fP on the target system and save the output: +.br +smemcap > memorycapture.tar +.IP 4. +Copy the output to another machine and run smem on it: +.br +smem -S memorycapture.tar .SH FILES .I /proc/$pid/cmdline @@ -149,8 +183,25 @@ Show pie graph. .PP .I /proc/version +.SH RESOURCES +Main Web Site: http://www.selenic.com/smem + +Source code repository: http://selenic.com/repo/smem + +Mailing list: http://selenic.com/mailman/listinfo/smem + .SH "SEE ALSO" -.BR free (1), pmap (1) +.BR free (1), +.BR pmap (1), +.BR proc (5), +.BR ps (1), +.BR top (1), +.BR vmstat (8) + +.SH COPYING +Copyright (C) 2008-2009 Matt Mackall. Free use of this software +is granted under the terms of the GNU General Public License +version 2 or later. .SH AUTHOR \fBsmem\fP was written by Matt Mackall. From 3d21e5daf30b683293ebe802bf55e214e71971b5 Mon Sep 17 00:00:00 2001 From: Johannes Stezenbach Date: Wed, 12 May 2010 15:59:24 -0500 Subject: [PATCH 20/58] smemcap: fix compile warnings --- smemcap.c | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/smemcap.c b/smemcap.c index d8af86f..c5eba4c 100644 --- a/smemcap.c +++ b/smemcap.c @@ -32,17 +32,17 @@ int writeheader(int destfd, const char *path, int mode, int uid, int gid, memset(header, 0, 512); sprintf(header, "%s", path); - sprintf(header + 100, "%07o\0", mode & 0777); - sprintf(header + 108, "%07o\0", uid); - sprintf(header + 116, "%07o\0", gid); - sprintf(header + 124, "%011o\0", size); - sprintf(header + 136, "%07o\0", mtime); + sprintf(header + 100, "%07o", mode & 0777); + sprintf(header + 108, "%07o", uid); + sprintf(header + 116, "%07o", gid); + sprintf(header + 124, "%011o", size); + sprintf(header + 136, "%07o", mtime); sprintf(header + 148, " %1d", type); /* fix checksum */ for (i = sum = 0; i < 512; i++) - sum += header[i]; - sprintf(header + 148, "%06o\0 %1d", sum, type); + sum += header[i]; + sprintf(header + 148, "%06o", sum); return write(destfd, header, 512); } @@ -91,7 +91,6 @@ int archivejoin(const char *sub, const char *name, int destfd) int main(int argc, char *argv[]) { - int fd; DIR *d; struct dirent *de; @@ -100,7 +99,7 @@ int main(int argc, char *argv[]) archivefile("version", 1); d = opendir("."); - while (de = readdir(d)) + while ((de = readdir(d))) if (de->d_name[0] >= '0' && de->d_name[0] <= '9') { writeheader(1, de->d_name, 0555, 0, 0, 0, 0, 5); archivejoin(de->d_name, "smaps", 1); From c08295242387d35262926db806dd7c83ac90e019 Mon Sep 17 00:00:00 2001 From: Yves Goergen Date: Wed, 16 Feb 2011 16:01:38 -0600 Subject: [PATCH 21/58] Avoid tracebacks on disappearing processes --- smem | 28 ++++++++++++++++++++++------ 1 file changed, 22 insertions(+), 6 deletions(-) diff --git a/smem b/smem index 66e7a91..9e751f1 100755 --- a/smem +++ b/smem @@ -35,20 +35,36 @@ class procdata(object): def version(self): return self._readlines('version')[0] def pidname(self, pid): - l = self._read('%d/stat' % pid) - return l[l.find('(') + 1: l.find(')')] + try: + l = self._read('%d/stat' % pid) + return l[l.find('(') + 1: l.find(')')] + except: + return '?' def pidcmd(self, pid): - c = self._read('%s/cmdline' % pid)[:-1] - return c.replace('\0', ' ') + try: + c = self._read('%s/cmdline' % pid)[:-1] + return c.replace('\0', ' ') + except: + return '?' def piduser(self, pid): - return self._stat('%d/cmdline' % pid).st_uid + try: + return self._stat('%d/cmdline' % pid).st_uid + except: + return -1 def pidgroup(self, pid): - return self._stat('%d/cmdline' % pid).st_gid + try: + return self._stat('%d/cmdline' % pid).st_gid + except: + return -1 def username(self, uid): + if uid == -1: + return '?' if uid not in self._ucache: self._ucache[uid] = pwd.getpwuid(uid)[0] return self._ucache[uid] def groupname(self, gid): + if gid == -1: + return '?' if gid not in self._gcache: self._gcache[gid] = pwd.getgrgid(gid)[0] return self._gcache[gid] From 455466fb67bdce37ee8ce6dd12184584115db546 Mon Sep 17 00:00:00 2001 From: Hynek Cernoch Date: Mon, 13 Dec 2010 22:33:05 +0100 Subject: [PATCH 22/58] Fixed bug in realmem option --- smem | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/smem b/smem index 9e751f1..da7f5fc 100755 --- a/smem +++ b/smem @@ -178,7 +178,9 @@ def fromunits(x): M=2**20, MB=2**20, G=2**30, GB=2**30) for k,v in s.items(): if x.endswith(k): - return int(float(x[:len(k)])*v) + return int(float(x[:-len(k)])*v) + sys.stderr.write("Memory size should be written with units, for example 1024M\n") + sys.exit(-1) def pidusername(pid): return src.username(src.piduser(pid)) From b0a3fae0e738cd03d05767a576dd4c3239529dc9 Mon Sep 17 00:00:00 2001 From: Hynek Cernoch Date: Mon, 13 Dec 2010 22:33:05 +0100 Subject: [PATCH 23/58] Add note about --realmem usage to manpage --- smem.8 | 1 + 1 file changed, 1 insertion(+) diff --git a/smem.8 b/smem.8 index 43418a4..f6ab5c1 100644 --- a/smem.8 +++ b/smem.8 @@ -48,6 +48,7 @@ source tree.) Amount of physical RAM. This lets smem detect the amount of memory used by firmware/hardware in the systemwide (\-w) output. If provided, it will also be used as the total memory size to base percentages on. +Example: --realmem=1024M .TP .BI "\-S " SOURCE ", \-\-source=" SOURCE From fc150ea960d6ce20bf35f602b413f664e9b185ae Mon Sep 17 00:00:00 2001 From: Hynek Cernoch Date: Mon, 13 Dec 2010 22:33:05 +0100 Subject: [PATCH 24/58] Give hint about uncompressed kernels --- smem | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/smem b/smem index da7f5fc..f7bfe0f 100755 --- a/smem +++ b/smem @@ -115,7 +115,16 @@ def kernelsize(): d = os.popen("size %s" % options.kernel).readlines()[1] _kernelsize = int(d.split()[3]) / 1024 except: - pass + try: + # try some heuristic to find gzipped part in kernel image + packedkernel = open(options.kernel).read() + pos = packedkernel.find('\x1F\x8B') + if pos >= 0 and pos < 25000: + sys.stderr.write("Maybe uncompressed kernel can be extracted by the command:\n" + " dd if=%s bs=1 skip=%d | gzip -d >%s.unpacked\n\n" % (options.kernel, pos, options.kernel)) + except: + pass + sys.stderr.write("Parameter '%s' should be an original uncompressed compiled kernel file.\n\n" % options.kernel) return _kernelsize def pidmaps(pid): From e2c5ced39acccb7b52a454313b79c6f6173e3b6e Mon Sep 17 00:00:00 2001 From: Hynek Cernoch Date: Mon, 13 Dec 2010 22:33:05 +0100 Subject: [PATCH 25/58] Clean up some tabs --- smem | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/smem b/smem index f7bfe0f..d9d9448 100755 --- a/smem +++ b/smem @@ -131,8 +131,8 @@ def pidmaps(pid): maps = {} start = None for l in src.mapdata(pid): - f = l.split() - if f[-1] == 'kB': + f = l.split() + if f[-1] == 'kB': maps[start][f[0][:-1].lower()] = int(f[1]) else: start, end = f[0].split('-') From 7b68171f07bd933aecf864692d8b5d92e7475064 Mon Sep 17 00:00:00 2001 From: Tim Bird Date: Wed, 16 Feb 2011 16:12:50 -0600 Subject: [PATCH 26/58] Fix bug in pie chart logic I was getting an error with pie charts on some systems with very small memory usage. $ smem -S data.tar --pie=command Traceback (most recent call last): File "/usr/local/bin/smem", line 636, in showpids() File "/usr/local/bin/smem", line 246, in showpids showtable(pt.keys(), fields, columns.split(), options.sort or 'pss') File "/usr/local/bin/smem", line 455, in showtable showpie(l, sort) File "/usr/local/bin/smem", line 498, in showpie while values and (t + values[-1 - c] < (tm * .02) or IndexError: list index out of range I traced it to a bug in showpie, where there's some confused usage of a list index and list popping. In showpie, c is used to index into the values in a while loop that removes entries from the end of a sorted list, and aggregates their values for use in an "other" entry, added to the list before display. Moving (and using) the index is wrong because the list is being chopped from the end as we go. This warps the value of 'other', but under normal circumstances would probably not be noticeable because these items have very small values. However, if several items are popped, and the list is very short, it can result in the list index error above. Also, truncating the values and labels in the subsequent conditional is redundant with the pop in the loop. Below is a patch to fix these problems. -- Tim --- smem | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) --- smem | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/smem b/smem index d9d9448..c09c134 100755 --- a/smem +++ b/smem @@ -521,15 +521,12 @@ def showpie(l, sort): s = sum(values) unused = tm - s t = 0 - c = 0 - while values and (t + values[-1 - c] < (tm * .02) or - values[-1 - c] < (tm * .005)): - c += 1 + while values and (t + values[-1] < (tm * .02) or + values[-1] < (tm * .005)): t += values.pop() labels.pop() - if c > 1: - values = values[:-c] - labels = labels[:-c] + + if t: values.append(t) labels.append('other') From 27fe66a83c453b9e17ff065fbc6d779fcec38f4c Mon Sep 17 00:00:00 2001 From: Matt Mackall Date: Thu, 26 May 2011 09:17:28 -0500 Subject: [PATCH 28/58] Add support for terabytes --- smem | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/smem b/smem index c09c134..9c3cb23 100755 --- a/smem +++ b/smem @@ -176,7 +176,7 @@ def units(x): s = '' if x == 0: return '0' - for s in ('', 'K', 'M', 'G'): + for s in ('', 'K', 'M', 'G', 'T'): if x < 1024: break x /= 1024.0 @@ -184,7 +184,8 @@ def units(x): def fromunits(x): s = dict(k=2**10, K=2**10, kB=2**10, KB=2**10, - M=2**20, MB=2**20, G=2**30, GB=2**30) + M=2**20, MB=2**20, G=2**30, GB=2**30, + T=2**40, TB=2**40) for k,v in s.items(): if x.endswith(k): return int(float(x[:-len(k)])*v) From 4cb161ea4587e5d67421a1b3dfed06fc80a2feaa Mon Sep 17 00:00:00 2001 From: Matt Mackall Date: Fri, 10 Jun 2011 08:58:26 -0500 Subject: [PATCH 29/58] read process uid/gid from task rather than cmdline --- smem | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/smem b/smem index 9c3cb23..d121d5d 100755 --- a/smem +++ b/smem @@ -48,12 +48,12 @@ class procdata(object): return '?' def piduser(self, pid): try: - return self._stat('%d/cmdline' % pid).st_uid + return self._stat('%d/task' % pid).st_uid except: return -1 def pidgroup(self, pid): try: - return self._stat('%d/cmdline' % pid).st_gid + return self._stat('%d/task' % pid).st_gid except: return -1 def username(self, uid): @@ -83,12 +83,12 @@ class tardata(procdata): def _readlines(self, f): return self.tar.extractfile(f).readlines() def piduser(self, p): - t = self.tar.getmember("%s/cmdline" % p) + t = self.tar.getmember("%s/task" % p) if t.uname: self._ucache[t.uid] = t.uname return t.uid def pidgroup(self, p): - t = self.tar.getmember("%s/cmdline" % p) + t = self.tar.getmember("%s/task" % p) if t.gname: self._gcache[t.gid] = t.gname return t.gid From 0279ad465aeb13094c0df3e272f8a7ad72d40d28 Mon Sep 17 00:00:00 2001 From: Matt Mackall Date: Wed, 17 Aug 2011 13:49:33 -0500 Subject: [PATCH 30/58] Actively detect PSS support Rather than checking the kernel version, look for PSS field when parsing smaps data and issue a warning. (based on a suggestion by Paul Townsend) --- smem | 18 ++++++++++-------- smem.8 | 3 ++- 2 files changed, 12 insertions(+), 9 deletions(-) diff --git a/smem b/smem index d121d5d..90a2fd6 100755 --- a/smem +++ b/smem @@ -10,6 +10,8 @@ import re, os, sys, pwd, grp, optparse, errno, tarfile +warned = False + class procdata(object): def __init__(self, source): self._ucache = {} @@ -128,11 +130,15 @@ def kernelsize(): return _kernelsize def pidmaps(pid): + global warned maps = {} start = None + seen = False for l in src.mapdata(pid): f = l.split() if f[-1] == 'kB': + if f[0].startswith('Pss'): + seen = True maps[start][f[0][:-1].lower()] = int(f[1]) else: start, end = f[0].split('-') @@ -144,6 +150,10 @@ def pidmaps(pid): offset=int(f[2], 16), device=f[3], inode=f[4], name=name) + if not seen and not warned: + sys.stderr.write('warning: kernel does not appear to support PSS measurement\n') + warned = True + if options.mapfilter: f = {} for m in maps: @@ -584,12 +594,6 @@ def showbar(l, columns, sort): pylab.legend([p[0] for p in pl], key) pylab.show() -def kernel_version_check(): - kernel_release = src.version().split()[2].split('-')[0] - if kernel_release < "2.6.27": - name = os.path.basename(sys.argv[0]) - sys.stderr.write(name + " requires a kernel >= 2.6.27\n") - sys.exit(-1) parser = optparse.OptionParser("%prog [options]") parser.add_option("-H", "--no-header", action="store_true", @@ -648,8 +652,6 @@ try: except: src = procdata(options.source) -kernel_version_check() - try: if options.mappings: showmaps() diff --git a/smem.8 b/smem.8 index f6ab5c1..6184a16 100644 --- a/smem.8 +++ b/smem.8 @@ -144,7 +144,8 @@ Show pie graph. \fBsmem\fP requires: .IP \(bu 3 -Linux kernel 2.6.27 or newer. +Linux kernel providing 'Pss' metric in /proc//smaps (generally +2.6.27 or newer). .IP \(bu Python 2.x (at least 2.4 or so). .IP \(bu From 0245f382ba22947229cb65cf455e4fcb6efe00c1 Mon Sep 17 00:00:00 2001 From: Matt Mackall Date: Wed, 17 Aug 2011 16:31:34 -0500 Subject: [PATCH 31/58] Use /usr/bin/env to locate Python --- smem | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/smem b/smem index 90a2fd6..0254baf 100755 --- a/smem +++ b/smem @@ -1,4 +1,4 @@ -#!/usr/bin/python +#!/usr/bin/env python # # smem - a tool for meaningful memory reporting # From bc481759297a58396f4ab85b3b298baa571de78f Mon Sep 17 00:00:00 2001 From: Paul Townsend Date: Wed, 17 Aug 2011 17:16:21 -0400 Subject: [PATCH 32/58] Catch KeyError on uid and gid conversion --- smem | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/smem b/smem index 0254baf..728377c 100755 --- a/smem +++ b/smem @@ -62,13 +62,19 @@ class procdata(object): if uid == -1: return '?' if uid not in self._ucache: - self._ucache[uid] = pwd.getpwuid(uid)[0] + try: + self._ucache[uid] = pwd.getpwuid(uid)[0] + except KeyError: + self._ucache[uid] = uid return self._ucache[uid] def groupname(self, gid): if gid == -1: return '?' if gid not in self._gcache: - self._gcache[gid] = pwd.getgrgid(gid)[0] + try: + self._gcache[gid] = pwd.getgrgid(gid)[0] + except KeyError: + self._gcache[gid] = gid return self._gcache[gid] class tardata(procdata): From 323e6491c4ec9cf7eef4694698ed8f0d8edc3e07 Mon Sep 17 00:00:00 2001 From: Paul Townsend Date: Mon, 22 Aug 2011 16:09:17 -0500 Subject: [PATCH 33/58] Grab uid info from /proc// stat --- smem | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/smem b/smem index 728377c..b8f990d 100755 --- a/smem +++ b/smem @@ -50,12 +50,12 @@ class procdata(object): return '?' def piduser(self, pid): try: - return self._stat('%d/task' % pid).st_uid + return self._stat('%d' % pid).st_uid except: return -1 def pidgroup(self, pid): try: - return self._stat('%d/task' % pid).st_gid + return self._stat('%d' % pid).st_gid except: return -1 def username(self, uid): @@ -91,12 +91,12 @@ class tardata(procdata): def _readlines(self, f): return self.tar.extractfile(f).readlines() def piduser(self, p): - t = self.tar.getmember("%s/task" % p) + t = self.tar.getmember("%d/" % p) if t.uname: self._ucache[t.uid] = t.uname return t.uid def pidgroup(self, p): - t = self.tar.getmember("%s/task" % p) + t = self.tar.getmember("%d/" % p) if t.gname: self._gcache[t.gid] = t.gname return t.gid From 76959e6b68e8dcbc28965b4f35a385c0a74811d6 Mon Sep 17 00:00:00 2001 From: Paul Townsend Date: Mon, 22 Aug 2011 16:09:51 -0500 Subject: [PATCH 34/58] Properly convert uid/gid to string --- smem | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/smem b/smem index b8f990d..c9496eb 100755 --- a/smem +++ b/smem @@ -65,7 +65,7 @@ class procdata(object): try: self._ucache[uid] = pwd.getpwuid(uid)[0] except KeyError: - self._ucache[uid] = uid + self._ucache[uid] = str(uid) return self._ucache[uid] def groupname(self, gid): if gid == -1: @@ -74,7 +74,7 @@ class procdata(object): try: self._gcache[gid] = pwd.getgrgid(gid)[0] except KeyError: - self._gcache[gid] = gid + self._gcache[gid] = str(gid) return self._gcache[gid] class tardata(procdata): From 5d5d2dd183b0222e80bc38931e56d97573747bb6 Mon Sep 17 00:00:00 2001 From: Paul Townsend Date: Mon, 22 Aug 2011 16:10:00 -0500 Subject: [PATCH 35/58] Sort the output by "rss" when no "Pss" present in smaps. --- smem | 2 ++ 1 file changed, 2 insertions(+) diff --git a/smem b/smem index c9496eb..af6ce94 100755 --- a/smem +++ b/smem @@ -159,6 +159,8 @@ def pidmaps(pid): if not seen and not warned: sys.stderr.write('warning: kernel does not appear to support PSS measurement\n') warned = True + if not options.sort: + options.sort = 'rss' if options.mapfilter: f = {} From 80ef1d7e5edc5137c971a977496b3ced23b53bd0 Mon Sep 17 00:00:00 2001 From: Paul Townsend Date: Mon, 22 Aug 2011 16:10:10 -0500 Subject: [PATCH 36/58] Store uid/gid/mtime for /proc directory capture --- smemcap.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/smemcap.c b/smemcap.c index c5eba4c..641c63b 100644 --- a/smemcap.c +++ b/smemcap.c @@ -93,6 +93,7 @@ int main(int argc, char *argv[]) { DIR *d; struct dirent *de; + struct stat s; chdir("/proc"); archivefile("meminfo", 1); @@ -101,7 +102,9 @@ int main(int argc, char *argv[]) d = opendir("."); while ((de = readdir(d))) if (de->d_name[0] >= '0' && de->d_name[0] <= '9') { - writeheader(1, de->d_name, 0555, 0, 0, 0, 0, 5); + stat (de->d_name, &s); + writeheader(1, de->d_name, 0555, s.st_uid, + s.st_gid, 0, s.st_mtime, 5); archivejoin(de->d_name, "smaps", 1); archivejoin(de->d_name, "cmdline", 1); archivejoin(de->d_name, "stat", 1); From d4406341a4ba19c47230cdc64396550e9599ab33 Mon Sep 17 00:00:00 2001 From: Matt Mackall Date: Wed, 30 Nov 2011 14:57:25 -0600 Subject: [PATCH 37/58] Be more forgiving of environment errors for memory and user views --- smem | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/smem b/smem index af6ce94..7b0eb7c 100755 --- a/smem +++ b/smem @@ -316,8 +316,8 @@ def maptotals(pids): t['pids'] += 1 seen[name] = 1 totals[name] = t - except: - raise + except EnvironmentError: + continue return totals def showmaps(): @@ -366,8 +366,8 @@ def usertotals(pids): maps = pidmaps(pid) if len(maps) == 0: continue - except: - raise + except EnvironmentError: + continue user = src.piduser(pid) if user not in totals: t = dict(size=0, rss=0, pss=0, shared_clean=0, From ac741c5ce97ebac7d993cd17bf181adbc27a9639 Mon Sep 17 00:00:00 2001 From: Matt Mackall Date: Mon, 5 Dec 2011 14:55:39 -0600 Subject: [PATCH 39/58] cache meminfo data --- smem | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/smem b/smem index 7b0eb7c..bbf6775 100755 --- a/smem +++ b/smem @@ -17,6 +17,7 @@ class procdata(object): self._ucache = {} self._gcache = {} self.source = source and source or "" + self._memdata = None def _list(self): return os.listdir(self.source + "/proc") def _read(self, f): @@ -33,7 +34,9 @@ class procdata(object): def mapdata(self, pid): return self._readlines('%s/smaps' % pid) def memdata(self): - return self._readlines('meminfo') + if self._memdata is None: + self._memdata = self._readlines('meminfo') + return self._memdata def version(self): return self._readlines('version')[0] def pidname(self, pid): From 9bba73fa6d1ece54147850183a9841c6568a1106 Mon Sep 17 00:00:00 2001 From: Paul Townsend Date: Mon, 5 Dec 2011 22:42:38 -0600 Subject: [PATCH 40/58] Count only filtered pids If '-t' is specified and a filter such as '-U me' is specified, the pid total displayed is the total number of pids instead of the number of filtered pids. --- smem | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/smem b/smem index bbf6775..1509954 100755 --- a/smem +++ b/smem @@ -269,7 +269,7 @@ def showpids(): return pidusername(p) fields = dict( - pid=('PID', lambda n: n, '% 5s', lambda x: len(p), + pid=('PID', lambda n: n, '% 5s', lambda x: len(pt), 'process ID'), user=('User', showuser, '%-8s', lambda x: len(dict.fromkeys(x)), 'owner of process'), From cf3da2ba2e1cc40cb346262a35cfa728be9b0f8d Mon Sep 17 00:00:00 2001 From: Paul Townsend Date: Thu, 8 Dec 2011 15:59:19 -0600 Subject: [PATCH 41/58] Fix percentage display for swap --- smem | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/smem b/smem index 1509954..506a394 100755 --- a/smem +++ b/smem @@ -216,11 +216,11 @@ def fromunits(x): def pidusername(pid): return src.username(src.piduser(pid)) -def showamount(a): +def showamount(a, total): if options.abbreviate: return units(a * 1024) elif options.percent: - return "%.2f%%" % (100.0 * a / totalmem()) + return "%.2f%%" % (100.0 * a / total) return a def filter(opt, arg, *sources): @@ -479,6 +479,9 @@ def showtable(rows, fields, columns, sort): if options.bar: columns.append(options.bar) + mt = totalmem() + st = memory()['swaptotal'] + for n in columns: if n not in fields: showfields(fields, n) @@ -486,7 +489,10 @@ def showtable(rows, fields, columns, sort): f = fields[n][2] if 'a' in f: - formatter.append(showamount) + if n == 'swap': + formatter.append(lambda x: showamount(x, st)) + else: + formatter.append(lambda x: showamount(x, mt)) f = f.replace('a', 's') else: formatter.append(lambda x: x) From 9c057328aeb634b70b20d5fe932d13a896528906 Mon Sep 17 00:00:00 2001 From: Matt Mackall Date: Mon, 29 Oct 2012 14:59:55 -0500 Subject: [PATCH 43/58] update COPYING for FSF address change --- COPYING | 41 ++++++++++++++++++++--------------------- 1 file changed, 20 insertions(+), 21 deletions(-) diff --git a/COPYING b/COPYING index d60c31a..d159169 100644 --- a/COPYING +++ b/COPYING @@ -1,12 +1,12 @@ - GNU GENERAL PUBLIC LICENSE - Version 2, June 1991 + GNU GENERAL PUBLIC LICENSE + Version 2, June 1991 - Copyright (C) 1989, 1991 Free Software Foundation, Inc. - 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + Copyright (C) 1989, 1991 Free Software Foundation, Inc., + 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. - Preamble + Preamble The licenses for most software are designed to take away your freedom to share and change it. By contrast, the GNU General Public @@ -15,7 +15,7 @@ software--to make sure the software is free for all its users. This General Public License applies to most of the Free Software Foundation's software and to any other program whose authors commit to using it. (Some other Free Software Foundation software is covered by -the GNU Library General Public License instead.) You can apply it to +the GNU Lesser General Public License instead.) You can apply it to your programs, too. When we speak of free software, we are referring to freedom, not @@ -55,8 +55,8 @@ patent must be licensed for everyone's free use or not licensed at all. The precise terms and conditions for copying, distribution and modification follow. - - GNU GENERAL PUBLIC LICENSE + + GNU GENERAL PUBLIC LICENSE TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 0. This License applies to any program or other work which contains @@ -110,7 +110,7 @@ above, provided that you also meet all of these conditions: License. (Exception: if the Program itself is interactive but does not normally print such an announcement, your work based on the Program is not required to print an announcement.) - + These requirements apply to the modified work as a whole. If identifiable sections of that work are not derived from the Program, and can be reasonably considered independent and separate works in @@ -168,7 +168,7 @@ access to copy from a designated place, then offering equivalent access to copy the source code from the same place counts as distribution of the source code, even though third parties are not compelled to copy the source along with the object code. - + 4. You may not copy, modify, sublicense, or distribute the Program except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense or distribute the Program is @@ -225,7 +225,7 @@ impose that choice. This section is intended to make thoroughly clear what is believed to be a consequence of the rest of this License. - + 8. If the distribution and/or use of the Program is restricted in certain countries either by patents or by copyrighted interfaces, the original copyright holder who places the Program under this License @@ -255,7 +255,7 @@ make exceptions for this. Our decision will be guided by the two goals of preserving the free status of all derivatives of our free software and of promoting the sharing and reuse of software generally. - NO WARRANTY + NO WARRANTY 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN @@ -277,9 +277,9 @@ YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. - END OF TERMS AND CONDITIONS - - How to Apply These Terms to Your New Programs + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it @@ -303,17 +303,16 @@ the "copyright" line and a pointer to where the full notice is found. MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA - + You should have received a copy of the GNU General Public License along + with this program; if not, write to the Free Software Foundation, Inc., + 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. Also add information on how to contact you by electronic and paper mail. If the program is interactive, make it output a short notice like this when it starts in an interactive mode: - Gnomovision version 69, Copyright (C) year name of author + Gnomovision version 69, Copyright (C) year name of author Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. This is free software, and you are welcome to redistribute it under certain conditions; type `show c' for details. @@ -336,5 +335,5 @@ necessary. Here is a sample; alter the names: This General Public License does not permit incorporating your program into proprietary programs. If your program is a subroutine library, you may consider it more useful to permit linking proprietary applications with the -library. If this is what you want to do, use the GNU Library General +library. If this is what you want to do, use the GNU Lesser General Public License instead of this License. From e570a14243c36b31cfef8e3732f91e6b209a3a06 Mon Sep 17 00:00:00 2001 From: Jani Monoses Date: Thu, 8 Nov 2012 14:25:42 -0600 Subject: [PATCH 44/58] drop trailing slashes when looking up user/group in tar snapshots --- smem | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/smem b/smem index 506a394..5be289b 100755 --- a/smem +++ b/smem @@ -94,12 +94,12 @@ class tardata(procdata): def _readlines(self, f): return self.tar.extractfile(f).readlines() def piduser(self, p): - t = self.tar.getmember("%d/" % p) + t = self.tar.getmember("%d" % p) if t.uname: self._ucache[t.uid] = t.uname return t.uid def pidgroup(self, p): - t = self.tar.getmember("%d/" % p) + t = self.tar.getmember("%d" % p) if t.gname: self._gcache[t.gid] = t.gname return t.gid From 2346c2b07580b95d7d741ecf01c20cae2ebd59d2 Mon Sep 17 00:00:00 2001 From: Matt Mackall Date: Fri, 22 Feb 2013 14:03:46 -0600 Subject: [PATCH 45/58] fix handling of new fields in smaps reported by Loic Minier --- smem | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/smem b/smem index 5be289b..7a3170d 100755 --- a/smem +++ b/smem @@ -149,7 +149,7 @@ def pidmaps(pid): if f[0].startswith('Pss'): seen = True maps[start][f[0][:-1].lower()] = int(f[1]) - else: + elif '-' in f[0] and ':' not in f[0]: # looks like a mapping range start, end = f[0].split('-') start = int(start, 16) name = "" From f8dee1586aa450b8df6f551ccc6d897700a1750c Mon Sep 17 00:00:00 2001 From: Lo?c Minier Date: Wed, 27 Mar 2013 20:01:07 -0700 Subject: [PATCH 46/58] avoid bogus warning on PSS measurement with empty smaps files --- smem | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/smem b/smem index 7a3170d..5b0ef8a 100755 --- a/smem +++ b/smem @@ -143,7 +143,9 @@ def pidmaps(pid): maps = {} start = None seen = False + empty = True for l in src.mapdata(pid): + empty = False f = l.split() if f[-1] == 'kB': if f[0].startswith('Pss'): @@ -159,7 +161,7 @@ def pidmaps(pid): offset=int(f[2], 16), device=f[3], inode=f[4], name=name) - if not seen and not warned: + if not empty and not seen and not warned: sys.stderr.write('warning: kernel does not appear to support PSS measurement\n') warned = True if not options.sort: From 9e788b26c663b631949af7970162b0fb835eaf80 Mon Sep 17 00:00:00 2001 From: Matt Mackall Date: Mon, 22 Apr 2013 17:21:03 -0500 Subject: [PATCH 48/58] drop unused import of grp --- smem | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/smem b/smem index 5b0ef8a..2d9a9c0 100755 --- a/smem +++ b/smem @@ -8,7 +8,7 @@ # the GNU General Public License version 2 or later, incorporated # herein by reference. -import re, os, sys, pwd, grp, optparse, errno, tarfile +import re, os, sys, pwd, optparse, errno, tarfile warned = False From 02bda05e5b2dcea76faa9c46dbfa7747a6b3a4ba Mon Sep 17 00:00:00 2001 From: Matt Mackall Date: Wed, 8 May 2013 14:21:28 -0500 Subject: [PATCH 49/58] handle division by zero with no swap Reported by Stefan Praszalowicz --- smem | 2 ++ 1 file changed, 2 insertions(+) diff --git a/smem b/smem index 2d9a9c0..0cbd925 100755 --- a/smem +++ b/smem @@ -222,6 +222,8 @@ def showamount(a, total): if options.abbreviate: return units(a * 1024) elif options.percent: + if total == 0: + return 'N/A' return "%.2f%%" % (100.0 * a / total) return a From 834dc815c8f3e446cdb70321d5072bb7ead50125 Mon Sep 17 00:00:00 2001 From: Matt Mackall Date: Thu, 22 May 2014 17:04:26 -0500 Subject: [PATCH 51/58] switch file() to open() --- smem | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/smem b/smem index 0cbd925..0222631 100755 --- a/smem +++ b/smem @@ -21,7 +21,7 @@ class procdata(object): def _list(self): return os.listdir(self.source + "/proc") def _read(self, f): - return file(self.source + '/proc/' + f).read() + return open(self.source + '/proc/' + f).read() def _readlines(self, f): return self._read(f).splitlines(True) def _stat(self, f): From 02dabc0b86da32dc0107168cfbe5517a9f75b700 Mon Sep 17 00:00:00 2001 From: Matt Mackall Date: Tue, 15 Jul 2014 01:33:26 -0500 Subject: [PATCH 52/58] rename filter to filters to make 2to3 happy --- smem | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/smem b/smem index 0222631..de221df 100755 --- a/smem +++ b/smem @@ -170,7 +170,7 @@ def pidmaps(pid): if options.mapfilter: f = {} for m in maps: - if not filter(options.mapfilter, m, lambda x: maps[x]['name']): + if not filters(options.mapfilter, m, lambda x: maps[x]['name']): f[m] = maps[m] return f @@ -227,7 +227,7 @@ def showamount(a, total): return "%.2f%%" % (100.0 * a / total) return a -def filter(opt, arg, *sources): +def filters(opt, arg, *sources): if not opt: return False @@ -252,8 +252,8 @@ def pidtotals(pid): def processtotals(pids): totals = {} for pid in pids: - if (filter(options.processfilter, pid, src.pidname, src.pidcmd) or - filter(options.userfilter, pid, pidusername)): + if (filters(options.processfilter, pid, src.pidname, src.pidcmd) or + filters(options.userfilter, pid, pidusername)): continue try: p = pidtotals(pid) @@ -301,8 +301,8 @@ def showpids(): def maptotals(pids): totals = {} for pid in pids: - if (filter(options.processfilter, pid, src.pidname, src.pidcmd) or - filter(options.userfilter, pid, pidusername)): + if (filters(options.processfilter, pid, src.pidname, src.pidcmd) or + filters(options.userfilter, pid, pidusername)): continue try: maps = pidmaps(pid) @@ -366,8 +366,8 @@ def showmaps(): def usertotals(pids): totals = {} for pid in pids: - if (filter(options.processfilter, pid, src.pidname, src.pidcmd) or - filter(options.userfilter, pid, pidusername)): + if (filters(options.processfilter, pid, src.pidname, src.pidcmd) or + filters(options.userfilter, pid, pidusername)): continue try: maps = pidmaps(pid) From 29fc357a3174511828e6b97110b3ced2d16d9d72 Mon Sep 17 00:00:00 2001 From: Cybjit Date: Tue, 18 Nov 2014 16:26:12 -0600 Subject: [PATCH 53/58] showfields: complain about sets --- smem | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/smem b/smem index de221df..c692f73 100755 --- a/smem +++ b/smem @@ -463,7 +463,9 @@ def showsystem(): showtable(range(len(l)), fields, columns.split(), options.sort or 'order') def showfields(fields, f): - if f != list: + if type(f) in (list, set): + print "unknown fields:", " ".join(f) + else: print "unknown field", f print "known fields:" for l in sorted(fields.keys()): @@ -486,11 +488,12 @@ def showtable(rows, fields, columns, sort): mt = totalmem() st = memory()['swaptotal'] - for n in columns: - if n not in fields: - showfields(fields, n) - sys.exit(-1) + missing = set(columns) - set(fields) + if len(missing) > 0: + showfields(fields, missing) + sys.exit(-1) + for n in columns: f = fields[n][2] if 'a' in f: if n == 'swap': From 272cb17cf9d37f4441e74ddfa36d1250c1d5a625 Mon Sep 17 00:00:00 2001 From: Cybjit Date: Tue, 18 Nov 2014 20:01:16 -0600 Subject: [PATCH 54/58] smem: allow column auto-sizing --- smem | 40 ++++++++++++++++++++++++++++++++++++++++ smem.8 | 4 ++++ 2 files changed, 44 insertions(+) diff --git a/smem b/smem index c692f73..2404835 100755 --- a/smem +++ b/smem @@ -471,6 +471,37 @@ def showfields(fields, f): for l in sorted(fields.keys()): print "%-8s %s" % (l, fields[l][-1]) +def autosize(columns, fields, rows): + colsizes = {} + for c in columns: + sizes = [1] + + if not options.no_header: + sizes.append(len(fields[c][0])) + + if (options.abbreviate or options.percent) and 'a' in fields[c][2]: + sizes.append(7) + else: + for r in rows: + sizes.append(len(str(fields[c][1](r)))) + + colsizes[c] = max(sizes) + + overflowcols = set(["command", "map"]) & set(columns) + if len(overflowcols) > 0: + overflowcol = overflowcols.pop() + totnoflow = sum(colsizes.values()) - colsizes[overflowcol] + try: + ttyrows, ttycolumns = os.popen('stty size', 'r').read().split() + ttyrows, ttycolumns = int(ttyrows), int(ttycolumns) + except: + ttyrows, ttycolumns = (24, 80) + maxflowcol = ttycolumns - totnoflow - len(columns) + maxflowcol = max(maxflowcol, 10) + colsizes[overflowcol] = min(colsizes[overflowcol], maxflowcol) + + return colsizes + def showtable(rows, fields, columns, sort): header = "" format = "" @@ -493,6 +524,11 @@ def showtable(rows, fields, columns, sort): showfields(fields, missing) sys.exit(-1) + if options.autosize: + colsizes = autosize(columns, fields, rows) + else: + colsizes = {} + for n in columns: f = fields[n][2] if 'a' in f: @@ -503,6 +539,8 @@ def showtable(rows, fields, columns, sort): f = f.replace('a', 's') else: formatter.append(lambda x: x) + if n in colsizes: + f = re.sub(r"[0-9]+", str(colsizes[n]), f) format += f + " " header += f % fields[n][0] + " " @@ -626,6 +664,8 @@ parser.add_option("-c", "--columns", type="str", help="columns to show") parser.add_option("-t", "--totals", action="store_true", help="show totals") +parser.add_option("-a", "--autosize", action="store_true", + help="size columns to fit terminal size") parser.add_option("-R", "--realmem", type="str", help="amount of physical RAM") diff --git a/smem.8 b/smem.8 index 6184a16..b71bd32 100644 --- a/smem.8 +++ b/smem.8 @@ -95,6 +95,10 @@ User filter regular expression. .SS OUTPUT FORMATTING +.TP +.B \-a, \-\-autosize +Size columns to fit terminal size. + .TP .BI "\-c " COLUMNS ", \-\-columns=" COLUMNS Columns to show. From afc19f79b684dc5541aa5a3cb0224ea8ef2a8cd0 Mon Sep 17 00:00:00 2001 From: Oren Tirosh Date: Fri, 15 May 2015 12:49:06 -0500 Subject: [PATCH 55/58] py3: stop using iterkeys() --- smem | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/smem b/smem index 2404835..b583835 100755 --- a/smem +++ b/smem @@ -240,7 +240,7 @@ def pidtotals(pid): maps = pidmaps(pid) t = dict(size=0, rss=0, pss=0, shared_clean=0, shared_dirty=0, private_clean=0, private_dirty=0, referenced=0, swap=0) - for m in maps.iterkeys(): + for m in maps: for k in t: t[k] += maps[m].get(k, 0) @@ -307,7 +307,7 @@ def maptotals(pids): try: maps = pidmaps(pid) seen = {} - for m in maps.iterkeys(): + for m in maps: name = maps[m]['name'] if name not in totals: t = dict(size=0, rss=0, pss=0, shared_clean=0, @@ -383,7 +383,7 @@ def usertotals(pids): else: t = totals[user] - for m in maps.iterkeys(): + for m in maps: for k in t: t[k] += maps[m].get(k, 0) From 0deddadf8807121e6a66dc6c0802b59ef8d81c01 Mon Sep 17 00:00:00 2001 From: Oren Tirosh Date: Fri, 15 May 2015 12:50:59 -0500 Subject: [PATCH 56/58] py3: add () to print statements It's a function call in 3.x and redundant parentheses around an expression on 2.x. This works fine as long as all prints have just a single argument. One print statement with two arguments was changed to use % formatting instead. --- smem | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/smem b/smem index b583835..9157ce0 100755 --- a/smem +++ b/smem @@ -464,12 +464,12 @@ def showsystem(): def showfields(fields, f): if type(f) in (list, set): - print "unknown fields:", " ".join(f) + print("unknown fields: " + " ".join(f)) else: - print "unknown field", f - print "known fields:" - for l in sorted(fields.keys()): - print "%-8s %s" % (l, fields[l][-1]) + print("unknown field %s" % f) + print("known fields:") + for l in sorted(fields): + print("%-8s %s" % (l, fields[l][-1])) def autosize(columns, fields, rows): colsizes = {} @@ -559,10 +559,10 @@ def showtable(rows, fields, columns, sort): return if not options.no_header: - print header + print(header) for k,r in l: - print format % tuple([f(v) for f,v in zip(formatter, r)]) + print(format % tuple([f(v) for f,v in zip(formatter, r)])) if options.totals: # totals @@ -574,8 +574,8 @@ def showtable(rows, fields, columns, sort): else: t.append("") - print "-" * len(header) - print format % tuple([f(v) for f,v in zip(formatter, t)]) + print("-" * len(header)) + print(format % tuple([f(v) for f,v in zip(formatter, t)])) def showpie(l, sort): try: From 8f2294477c1b3c62ec7e7384147c487d71a7ba81 Mon Sep 17 00:00:00 2001 From: Oren Tirosh Date: Fri, 15 May 2015 12:52:02 -0500 Subject: [PATCH 57/58] py3: do not use "except Exception, e:" syntax If catching the exception is done with "except Exception as e:" this will work only from 2.6. To maintain backward compatibility with 2.4 I have used sys.exc_info(). A bit of a wart, but it works. --- smem | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/smem b/smem index 9157ce0..7020a27 100755 --- a/smem +++ b/smem @@ -725,7 +725,8 @@ try: showsystem() else: showpids() -except IOError, e: +except IOError: + _, e, _ = sys.exc_info() if e.errno == errno.EPIPE: pass except KeyboardInterrupt: